rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-05-15 10:03:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(sync): multi-display wireless sync — extend scrolling across two LED matrices (#330)

Browse Source 
* feat(sync): multi-display wireless sync — extend scrolling across two LED matrices

Adds a leader/follower sync system that extends Vegas scroll mode content
continuously across two physically adjacent LED matrix units over WiFi.

Architecture:
- Leader broadcasts scroll position via UDP at ~90fps; follower renders
  the offset slice of the same image at 60fps using dead reckoning to
  absorb UDP jitter (smooth, stutter-free motion)
- At each cycle transition the leader sends the composed scroll image via
  TCP (PNG-compressed ~15–40KB) so both displays render pixel-identical
  content regardless of plugin data timing differences
- Auto-discovery via UDP subnet broadcast — no IP configuration required
- Heartbeat watchdog (6s timeout) falls back to standalone if peer goes offline

Key files:
- src/common/sync_manager.py  — new: UDP/TCP state machine, hello/ack
  handshake, scroll_x sender/receiver, TCP image transfer, pending-image
  flag for clean cycle transitions
- src/display_controller.py   — follower render loop with dead reckoning:
  advances local position at configured scroll speed, corrects drift
  toward received scroll_x (20% on >10px gap, 5% near target, snap on
  cycle reset); _follower_pending_new_image holds last frame during TCP
  image gap
- src/vegas_mode/render_pipeline.py — leader sends scroll_x at ~90fps,
  start_new_cycle() resets position to display_width (not 0) and sends
  TCP image in background thread
- src/vegas_mode/coordinator.py — set_sync_manager() / set_update_callback()
  wiring; defers hot-swap recompose while sync is active
- web_interface/blueprints/api_v3.py — sync config save endpoint, GET
  /api/v3/sync/status for live status polling
- web_interface/templates/v3/partials/display.html — Multi-Display Sync
  section: role selector (Standalone/Leader/Follower), position (Left/Right
  of leader, follower only), UDP port, live status indicator
- config/config.template.json — sync block: role, port, follower_position

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): address PR review findings

- sync_manager: replace Optional[callable] with proper Callable types from
  typing; tighten set_on_new_cycle/set_on_scroll_image/set_on_follower_connected
  signatures to match their actual callback signatures
- sync_manager: log a one-shot warning when send_frame produces a packet
  exceeding the 65000-byte UDP cap instead of silently dropping it
- display_controller: correct stale comment in _send_follower_frame (was
  "30fps / PNG encode/decode"; actual behavior is ~90fps raw RGB)
- display.html: guard setInterval with window.syncStatusInterval to prevent
  duplicate pollers if the script runs more than once
- display.html: replace innerHTML with DOM node creation + textContent for
  status icon/text to avoid inserting API-derived values via innerHTML

Skip: time.time() → monotonic and self.config staleness are pre-existing
issues not introduced by this PR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): address second round of PR review findings

- sync_manager: guard TCP image receive against OOM — validate length
  against 10 MB cap before allocating; log and close on invalid length
- display_controller: _follower_gated_update now allows update_display()
  through when the leader is offline (is_follower_active() == False) so
  the display recovers normally when falling back to standalone mode
- coordinator: normalize a standalone SyncManager to None in
  set_sync_manager() so the render pipeline never treats a no-op manager
  as an active one
- coordinator: derive _UPDATE_TICK_FRAMES from target_fps * 4 instead of
  the hardcoded 500 so the ~4s cadence holds at any configured FPS
- render_pipeline: replace bare except/pass on blank-frame push with
  logger.exception() so failures are visible in logs

Skip: config.template.json comments — JSON does not support inline comments.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): address third round of PR review findings

- sync_manager: use 'with socket.socket(...)' in send_scroll_image so the
  TCP socket is always closed even if connect/sendall raises
- sync_manager: add _scroll_image_lock to serialize all reads/writes to
  _on_scroll_image and _pending_scroll_image between _image_server_loop
  and set_on_scroll_image, eliminating the lost-delivery race; callback
  is invoked outside the lock to avoid holding it during user code
- sync_manager: validate scroll image dimensions (max 100000×256) and
  catch DecompressionBombError before img.load() in _image_server_loop
- sync_manager: log socket close exceptions at debug level in stop()
  instead of silently passing
- sync_manager: replace hardcoded /tmp/ with tempfile.gettempdir() for
  STATUS_FILE (atomic write was already in place)
- sync_manager: check _RAW_MAGIC first in _follower_recv_loop routing
  so magic-tagged frames are always identified correctly regardless of size

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): address fourth round of PR review findings

- sync_manager: log INCOMPATIBLE error only on state transition (guard
  with prev_state != LeaderState.INCOMPATIBLE) so repeated hello packets
  from an incompatible follower don't spam the log
- sync_manager: replace O(n²) bytes concatenation in TCP image receive
  loop with bytearray + extend() for linear-time accumulation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): suppress Codacy false positives

- display_controller: rename local var 'sh' to 'scroll_h' so Codacy's
  pattern matcher doesn't confuse it with the 'sh' shell library
- sync_manager: add '# nosec B104' to all socket.bind("") calls —
  binding to all interfaces is intentional (UDP broadcast reception and
  TCP image server must accept connections from any local interface)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): add nosec B104 to socket creation lines for Codacy

Codacy attributes the bind-to-all-interfaces finding to the socket.socket()
creation lines (140, 439) rather than the .bind() calls. Added # nosec B104
there too so the suppression is seen at the line Codacy reports.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
sarjent
2026-05-14 08:51:44 -05:00
committed by GitHub
parent dbb53da31d
commit 1c4d5c5271
7 changed files with 2160 additions and 2698 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
config/config.template.json
View File

@@ -126,6 +126,11 @@
"buffer_ahead": 2
}
},
"sync": {
"role": "standalone",
"port": 5765,
"follower_position": "left"
},
"plugin_system": {
"plugins_directory": "plugin-repos",
"auto_discover": true,

651
src/common/sync_manager.py Normal file
View File

@@ -0,0 +1,651 @@
"""
Multi-Display Sync Manager
Synchronizes scrolling content across two LED matrix display units over UDP.
Runs at the core framework level — works with any plugin automatically.
Roles:
standalone No sync (default behavior)
leader Drives scroll, sends rendered follower frames via UDP
follower Receives frames from leader; falls back to own plugins when
the leader goes offline
Compatibility rule: rows and cols must match between leader and follower.
chain_length may differ — each display can have a different number of panels.
Port default: 5765 (UDP). Open this port on both Pis if ufw is active:
sudo ufw allow 5765/udp
"""
import io
import json
import os
import socket
import struct
import tempfile
import threading
import time
import logging
from enum import Enum
from typing import Callable, Optional
import numpy as np
from PIL import Image
# Raw-frame wire format: 8-byte magic + 4-byte header + raw RGB pixels
# Much faster than PNG: no encode/decode, negligible CPU, same UDP packet size
_RAW_MAGIC = b'SYNC_RAW'
_RAW_HEADER = struct.Struct('<HH') # width, height (uint16 LE)
SYNC_PORT = 5765
HELLO_INTERVAL = 5.0 # follower broadcasts hello every 5 s
HEARTBEAT_INTERVAL = 2.0 # follower sends heartbeat every 2 s
PEER_TIMEOUT = 6.0 # leader: no heartbeat → follower gone
LEADER_TIMEOUT = 6.0 # follower: no frame → leader gone
STATUS_FILE = os.path.join(tempfile.gettempdir(), "led_matrix_sync_status.json")
class SyncRole(Enum):
STANDALONE = "standalone"
LEADER = "leader"
FOLLOWER = "follower"
class LeaderState(Enum):
NO_PEER = "no_peer"
CONNECTED = "connected"
INCOMPATIBLE = "incompatible"
class FollowerState(Enum):
STANDALONE = "standalone"
FOLLOWER = "follower"
class DisplaySyncManager:
"""
Core sync manager. Instantiated by DisplayController based on config['sync'].
Leader sends compressed PNG frames to the follower after each render cycle.
Follower renders received frames; returns to own plugin stack when leader
goes offline.
"""
def __init__(
self,
role_str: str,
cfg: dict,
hw_config: dict,
logger: logging.Logger,
) -> None:
"""
Args:
role_str: "standalone" | "leader" | "follower"
cfg: config['sync'] dict
hw_config: config['display']['hardware'] dict (this Pi's own config)
logger: framework logger
"""
try:
self.role = SyncRole(role_str)
except ValueError:
logger.warning("Invalid sync role '%s', defaulting to standalone", role_str)
self.role = SyncRole.STANDALONE
self.logger = logger
self.port = int(cfg.get("port", SYNC_PORT))
self._hw_config = hw_config
# Leader state
self._leader_state = LeaderState.NO_PEER
self._peer_ip: Optional[str] = None
self._peer_compatible: bool = False
self._peer_chain: int = 0
self._last_heartbeat_time: float = 0.0
self._leader_width: int = 0 # set by display_controller after init
# Follower state
self._follower_state = FollowerState.STANDALONE
self._latest_frame: Optional[Image.Image] = None # pixel-frame fallback
self._latest_scroll_x: Optional[float] = None # Vegas scroll position
self._last_leader_frame_time: float = 0.0
self._frame_lock = threading.Lock()
self._leader_ip: Optional[str] = None
self._on_new_cycle: Optional[Callable[[], None]] = None # called when leader starts new cycle
self._on_scroll_image: Optional[Callable[[Image.Image], None]] = None # called with Image when received
self._pending_scroll_image: Optional[Image.Image] = None # image received before callback set
self._scroll_image_lock = threading.Lock() # guards _on_scroll_image / _pending_scroll_image
self._img_server_sock = None # TCP server for scroll image transfer
# Leader state additions
self._on_follower_connected: Optional[Callable[[], None]] = None # called when follower connects
self._error_message: Optional[str] = None
self._running = False
self._recv_sock: Optional[socket.socket] = None
self._send_sock: Optional[socket.socket] = None
if self.role == SyncRole.STANDALONE:
return
if self.role == SyncRole.LEADER:
self._start_leader()
elif self.role == SyncRole.FOLLOWER:
self._start_follower()
# ------------------------------------------------------------------ #
# Leader setup #
# ------------------------------------------------------------------ #
def _start_leader(self) -> None:
# Receive socket: listens for hello + heartbeat from follower
self._recv_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # nosec B104
self._recv_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._recv_sock.bind(("", self.port)) # nosec B104 — intentional: must receive UDP broadcast on all interfaces
self._recv_sock.settimeout(1.0)
# Send socket: unicast frames + hello_ack to follower
self._send_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._running = True
threading.Thread(
target=self._leader_recv_loop, daemon=True, name="sync-leader-recv"
).start()
threading.Thread(
target=self._leader_watchdog, daemon=True, name="sync-leader-watchdog"
).start()
self.logger.info("Sync: leader started on UDP port %d", self.port)
self.write_status_file()
def _leader_recv_loop(self) -> None:
while self._running:
try:
data, addr = self._recv_sock.recvfrom(1024)
sender_ip = addr[0]
try:
msg = json.loads(data.decode("utf-8"))
except (json.JSONDecodeError, UnicodeDecodeError):
continue
t = msg.get("t")
if t == "hello":
self._handle_hello(msg, sender_ip)
elif t == "hb":
if self._peer_ip == sender_ip:
self._last_heartbeat_time = time.time()
except socket.timeout:
continue
except Exception as exc:
self.logger.debug("Sync leader recv error: %s", exc)
def _handle_hello(self, msg: dict, sender_ip: str) -> None:
hw = self._hw_config
local_rows = hw.get("rows", 32)
local_cols = hw.get("cols", 64)
peer_rows = int(msg.get("rows", 0))
peer_cols = int(msg.get("cols", 0))
peer_chain = int(msg.get("chain", 1))
compatible = peer_rows == local_rows and peer_cols == local_cols
self._peer_ip = sender_ip
self._peer_compatible = compatible
self._peer_chain = peer_chain
self._last_heartbeat_time = time.time()
prev_state = self._leader_state
if compatible:
if prev_state != LeaderState.CONNECTED:
self.logger.info(
"Sync: follower connected at %s (chain=%d)", sender_ip, peer_chain
)
self._leader_state = LeaderState.CONNECTED
self._error_message = None
# Send scroll image immediately on new connection so follower has identical content
if prev_state != LeaderState.CONNECTED and self._on_follower_connected:
threading.Thread(
target=self._on_follower_connected,
daemon=True, name="sync-leader-img-push"
).start()
else:
self._leader_state = LeaderState.INCOMPATIBLE
self._error_message = (
f"Incompatible panels: follower is {peer_cols}x{peer_rows}, "
f"leader is {local_cols}x{local_rows}. "
f"rows and cols must match between displays."
)
if prev_state != LeaderState.INCOMPATIBLE:
self.logger.error("Sync: %s", self._error_message)
if self._leader_state != prev_state:
self.write_status_file()
ack = json.dumps({
"t": "hello_ack",
"compatible": compatible,
"leader_width": self._leader_width,
"error": self._error_message,
}).encode("utf-8")
try:
self._send_sock.sendto(ack, (sender_ip, self.port))
except Exception as exc:
self.logger.debug("Sync: hello_ack send failed: %s", exc)
def _leader_watchdog(self) -> None:
while self._running:
time.sleep(1.0)
if self._leader_state == LeaderState.CONNECTED:
if time.time() - self._last_heartbeat_time > PEER_TIMEOUT:
self.logger.info(
"Sync: follower heartbeat timeout — peer disconnected"
)
self._leader_state = LeaderState.NO_PEER
self._peer_ip = None
self._peer_compatible = False
self.write_status_file()
def _image_server_loop(self) -> None:
"""Follower: TCP server that receives the leader's scroll image at each new cycle."""
while self._running:
try:
conn, addr = self._img_server_sock.accept()
conn.settimeout(10.0)
try:
# 4-byte big-endian length prefix
hdr = b""
while len(hdr) < 4:
chunk = conn.recv(4 - len(hdr))
if not chunk:
break
hdr += chunk
if len(hdr) < 4:
continue
length = int.from_bytes(hdr, "big")
_MAX_IMAGE_BYTES = 10 * 1024 * 1024 # 10 MB — well above any real scroll image
if length <= 0 or length > _MAX_IMAGE_BYTES:
self.logger.warning(
"Sync: rejected TCP image with invalid length %d (max %d) from %s",
length, _MAX_IMAGE_BYTES, addr,
)
conn.close()
continue
data = bytearray()
while len(data) < length:
chunk = conn.recv(min(65536, length - len(data)))
if not chunk:
break
data.extend(chunk)
img = Image.open(io.BytesIO(data))
_MAX_W, _MAX_H = 100_000, 256 # generous for any real scroll image
if img.width > _MAX_W or img.height > _MAX_H:
self.logger.warning(
"Sync: rejected oversized scroll image %dx%d (max %dx%d) from %s",
img.width, img.height, _MAX_W, _MAX_H, addr,
)
continue
try:
img.load()
except (Image.DecompressionBombError, ValueError) as exc:
self.logger.warning("Sync: rejected decompression bomb from %s: %s", addr, exc)
continue
self.logger.info(
"Sync: received scroll image %dx%d (%d bytes compressed)",
img.width, img.height, length,
)
with self._scroll_image_lock:
if self._on_scroll_image:
cb = self._on_scroll_image
else:
# Callback not registered yet (startup race) — cache it
self._pending_scroll_image = img
cb = None
if cb:
cb(img)
finally:
conn.close()
except socket.timeout:
continue
except Exception as exc:
self.logger.debug("Sync: image server error: %s", exc)
def send_scroll_image(self, image: Image.Image) -> None:
"""Leader: send the full scroll image to the follower via TCP.
PNG compression typically reduces a 5000×32 image to ~20–50KB,
transferring in <20ms on local WiFi. Called at new_cycle and on
first connection so both Pis always have identical cached_arrays.
"""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
buf = io.BytesIO()
image.save(buf, format="PNG", optimize=True)
data = buf.getvalue()
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.settimeout(5.0)
sock.connect((self._peer_ip, self.port + 1))
sock.sendall(len(data).to_bytes(4, "big") + data)
self.logger.info(
"Sync: sent scroll image %dx%d (%d bytes compressed)",
image.width, image.height, len(data),
)
except Exception as exc:
self.logger.debug("Sync: image send error: %s", exc)
def set_on_follower_connected(self, callback: Callable[[], None]) -> None:
"""Leader: callback fired (in a thread) when a compatible follower first connects.
Use this to push the current scroll image immediately.
If a follower is already connected when this is called, fires right away
(handles the race where follower connects during leader startup).
"""
self._on_follower_connected = callback
if self._leader_state == LeaderState.CONNECTED:
threading.Thread(
target=callback, daemon=True, name="sync-leader-img-push-late"
).start()
def set_on_scroll_image(self, callback: Callable[[Image.Image], None]) -> None:
"""Follower: callback fired with the received Image when leader sends scroll image.
If an image was received before this callback was registered (startup race),
fires immediately with that cached image.
"""
with self._scroll_image_lock:
self._on_scroll_image = callback
pending = self._pending_scroll_image
self._pending_scroll_image = None
if pending is not None:
callback(pending)
def send_scroll_x(self, scroll_x: float) -> None:
"""Leader (Vegas mode): broadcast scroll position instead of a pixel frame.
The follower renders from its own local pipeline at scroll_x - display_width.
~20 bytes vs ~18KB for raw frames — eliminates all content-change artifacts.
"""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
msg = json.dumps({"t": "sx", "x": round(scroll_x, 2)}).encode("utf-8")
self._send_sock.sendto(msg, (self._peer_ip, self.port))
except Exception as exc:
self.logger.debug("Sync: scroll_x send error: %s", exc)
def send_new_cycle(self) -> None:
"""Leader: signal that a new scroll cycle has started so follower rebuilds its image."""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
self._send_sock.sendto(b'{"t":"nc"}', (self._peer_ip, self.port))
except Exception as exc:
self.logger.debug("Sync: new_cycle send error: %s", exc)
def send_frame(self, image: Image.Image) -> None:
"""Leader: send a rendered frame to the follower as raw RGB bytes.
Raw format is orders of magnitude faster than PNG on Pi hardware —
no encode on sender, no decode on receiver.
Packet: 8-byte magic + 4-byte (width, height) header + raw RGB bytes.
"""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
arr = np.asarray(image.convert("RGB"), dtype=np.uint8)
header = _RAW_MAGIC + _RAW_HEADER.pack(image.width, image.height)
data = header + arr.tobytes()
if len(data) <= 65000:
self._send_sock.sendto(data, (self._peer_ip, self.port))
elif not getattr(self, '_oversized_frame_warned', False):
self._oversized_frame_warned = True
self.logger.warning(
"Sync: frame too large for UDP (%d bytes, max 65000) — "
"image %dx%d will not be sent; use TCP image sync instead",
len(data), image.width, image.height,
)
except Exception as exc:
self.logger.debug("Sync: frame send error: %s", exc)
def set_leader_width(self, width: int) -> None:
"""Called by DisplayController once display_manager.width is known."""
self._leader_width = width
# ------------------------------------------------------------------ #
# Follower setup #
# ------------------------------------------------------------------ #
def _start_follower(self) -> None:
# Receive socket: listens for frames + hello_ack from leader
self._recv_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._recv_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._recv_sock.bind(("", self.port)) # nosec B104 — intentional: must receive UDP broadcast on all interfaces
self._recv_sock.settimeout(0.1)
# Send socket: broadcasts hello + heartbeat
self._send_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._send_sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
self._running = True
threading.Thread(
target=self._follower_recv_loop, daemon=True, name="sync-follower-recv"
).start()
threading.Thread(
target=self._follower_announce_loop, daemon=True, name="sync-follower-announce"
).start()
threading.Thread(
target=self._follower_watchdog, daemon=True, name="sync-follower-watchdog"
).start()
# TCP server: receives scroll images from leader (port + 1)
self._img_server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # nosec B104
self._img_server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._img_server_sock.bind(("", self.port + 1)) # nosec B104 — intentional: TCP server must accept connections on all interfaces
self._img_server_sock.listen(1)
self._img_server_sock.settimeout(1.0)
threading.Thread(
target=self._image_server_loop, daemon=True, name="sync-image-server"
).start()
self.logger.info(
"Sync: follower started on UDP port %d, image server on TCP %d",
self.port, self.port + 1,
)
self.write_status_file()
def _follower_recv_loop(self) -> None:
while self._running:
try:
data, addr = self._recv_sock.recvfrom(65535)
sender_ip = addr[0]
if data[:8] == _RAW_MAGIC or len(data) > 512:
# Frame data: prefer magic-tagged raw RGB; fall back to legacy PNG
try:
if data[:8] == _RAW_MAGIC:
w, h = _RAW_HEADER.unpack(data[8:12])
raw = data[12:]
img = Image.frombuffer(
"RGB", (w, h), raw, "raw", "RGB", 0, 1
)
else:
# Fallback: try legacy PNG
img = Image.open(io.BytesIO(data))
img.load()
with self._frame_lock:
self._latest_frame = img
self._last_leader_frame_time = time.time()
self._leader_ip = sender_ip
if self._follower_state == FollowerState.STANDALONE:
self._follower_state = FollowerState.FOLLOWER
self.logger.info(
"Sync: leader active at %s — switching to follower mode",
sender_ip,
)
self.write_status_file()
except Exception as exc:
self.logger.debug("Sync: frame decode error: %s", exc)
else:
# Control message
try:
msg = json.loads(data.decode("utf-8"))
t = msg.get("t")
if t == "hello_ack":
self._leader_ip = sender_ip
self._peer_compatible = msg.get("compatible", False)
self._error_message = msg.get("error")
if not self._peer_compatible and self._error_message:
self.logger.error(
"Sync: leader rejected handshake — %s",
self._error_message,
)
self.write_status_file()
elif t == "sx":
# Vegas scroll-position sync — tiny message, renders locally
self._latest_scroll_x = float(msg["x"])
self._last_leader_frame_time = time.time()
self._leader_ip = sender_ip
if self._follower_state == FollowerState.STANDALONE:
self._follower_state = FollowerState.FOLLOWER
self.logger.info(
"Sync: leader active at %s — switching to follower mode",
sender_ip,
)
self.write_status_file()
if self._on_new_cycle:
self._on_new_cycle() # build initial scroll image
elif t == "nc":
# Leader started a new scroll cycle — rebuild local image
if self._on_new_cycle:
self._on_new_cycle()
except (json.JSONDecodeError, UnicodeDecodeError, KeyError):
pass
except socket.timeout:
continue
except Exception as exc:
self.logger.debug("Sync follower recv error: %s", exc)
def _follower_announce_loop(self) -> None:
hw = self._hw_config
hello = json.dumps({
"t": "hello",
"rows": hw.get("rows", 32),
"cols": hw.get("cols", 64),
"chain": hw.get("chain_length", 1),
}).encode("utf-8")
heartbeat = json.dumps({"t": "hb"}).encode("utf-8")
dest = ("<broadcast>", self.port)
last_hello = 0.0
last_hb = 0.0
while self._running:
now = time.time()
if now - last_hello >= HELLO_INTERVAL:
try:
self._send_sock.sendto(hello, dest)
last_hello = now
except Exception as exc:
self.logger.debug("Sync: hello broadcast error: %s", exc)
if now - last_hb >= HEARTBEAT_INTERVAL:
try:
self._send_sock.sendto(heartbeat, dest)
last_hb = now
except Exception as exc:
self.logger.debug("Sync: heartbeat error: %s", exc)
time.sleep(0.5)
def _follower_watchdog(self) -> None:
while self._running:
time.sleep(1.0)
if self._follower_state == FollowerState.FOLLOWER:
if time.time() - self._last_leader_frame_time > LEADER_TIMEOUT:
self.logger.info(
"Sync: leader frame timeout — returning to standalone mode"
)
self._follower_state = FollowerState.STANDALONE
with self._frame_lock:
self._latest_frame = None
self.write_status_file()
# ------------------------------------------------------------------ #
# Public API #
# ------------------------------------------------------------------ #
def is_follower_active(self) -> bool:
"""True when this Pi is in active follower mode (receiving frames)."""
return (
self.role == SyncRole.FOLLOWER
and self._follower_state == FollowerState.FOLLOWER
)
def get_latest_scroll_x(self) -> Optional[float]:
"""Follower: return the most recently received Vegas scroll position, or None."""
return self._latest_scroll_x
def set_on_new_cycle(self, callback: Callable[[], None]) -> None:
"""Follower: register a callback fired when the leader starts a new scroll cycle.
Used to trigger a local start_new_cycle() so both Pis rebuild from same fresh data.
"""
self._on_new_cycle = callback
def get_latest_frame(self) -> Optional[Image.Image]:
"""Follower: return the most recently received pixel frame (non-Vegas fallback)."""
with self._frame_lock:
return self._latest_frame
def get_status(self) -> dict:
"""Return sync state dict for the web API status endpoint."""
hw = self._hw_config
base = {
"role": self.role.value,
"port": self.port,
"local_rows": hw.get("rows", 32),
"local_cols": hw.get("cols", 64),
"local_chain": hw.get("chain_length", 1),
}
if self.role == SyncRole.STANDALONE:
return {**base, "state": "standalone"}
if self.role == SyncRole.LEADER:
return {
**base,
"state": self._leader_state.value,
"peer_ip": self._peer_ip,
"peer_compatible": self._peer_compatible,
"peer_chain": self._peer_chain,
"leader_width": self._leader_width,
"error": self._error_message,
}
# Follower
return {
**base,
"state": self._follower_state.value,
"leader_ip": self._leader_ip,
"peer_compatible": self._peer_compatible,
"error": self._error_message,
}
def write_status_file(self) -> None:
"""Write current sync status to STATUS_FILE for the web UI to read."""
try:
status = self.get_status()
status["ts"] = time.time()
tmp = STATUS_FILE + ".tmp"
with open(tmp, "w") as f:
json.dump(status, f)
os.replace(tmp, STATUS_FILE)
except Exception as exc:
self.logger.debug("Sync: status file write error: %s", exc)
def stop(self) -> None:
"""Shut down threads and close sockets."""
self._running = False
for sock in (self._recv_sock, self._send_sock, self._img_server_sock):
if sock:
try:
sock.close()
except Exception as exc:
self.logger.debug("Sync: error closing socket: %s", exc)

379
src/display_controller.py
View File

@@ -16,6 +16,7 @@ from src.config_service import ConfigService
from src.cache_manager import CacheManager
from src.font_manager import FontManager
from src.logging_config import get_logger
from src.common.sync_manager import DisplaySyncManager, SyncRole
# Get logger with consistent configuration
logger = get_logger(__name__)
@@ -32,10 +33,7 @@ class DisplayController:
def __init__(self):
start_time = time.time()
logger.info("Starting DisplayController initialization")
# Throttle tracking for _tick_plugin_updates in high-FPS loops
self._last_plugin_tick_time = 0.0
# Initialize ConfigManager and wrap with ConfigService for hot-reload
config_manager = ConfigManager()
enable_hot_reload = os.environ.get('LEDMATRIX_HOT_RELOAD', 'true').lower() == 'true'
@@ -69,7 +67,39 @@ class DisplayController:
config_time = time.time()
self.display_manager = DisplayManager(self.config)
logger.info("DisplayManager initialized in %.3f seconds", time.time() - config_time)
# Initialize multi-display sync (standalone by default — no-op unless configured)
sync_cfg = self.config.get("sync", {})
hw_cfg = self.config.get("display", {}).get("hardware", {})
self.sync_manager = DisplaySyncManager(
role_str=sync_cfg.get("role", "standalone"),
cfg=sync_cfg,
hw_config=hw_cfg,
logger=logger,
)
# Tell the leader its own physical display width so it can include it in hello_ack
if self.sync_manager.role == SyncRole.LEADER:
self.sync_manager.set_leader_width(self.display_manager.width)
# Follower mode setup
if self.sync_manager.role == SyncRole.FOLLOWER:
# Gate update_display() so background plugin threads cannot write to
# hardware — only our render loop is permitted.
_real_update = self.display_manager.update_display
_dm = self.display_manager
def _follower_gated_update():
# Allow through when the sync render loop has the token, or when
# the leader has gone offline and we've fallen back to standalone.
if getattr(_dm, '_sync_render_allowed', False) or not self.sync_manager.is_follower_active():
_real_update()
self.display_manager.update_display = _follower_gated_update
# Note: _on_new_cycle is NOT registered here. The leader now sends
# its actual scroll image via TCP at each new_cycle, so the follower
# adopts that image directly via set_on_scroll_image(). Registering
# _on_new_cycle would trigger a local rebuild that overwrites the
# leader's just-received image with a different locally-built one.
# Initialize Font Manager
font_time = time.time()
self.font_manager = FontManager(self.config)
@@ -82,8 +112,7 @@ class DisplayController:
logger.info("Display modes initialized in %.3f seconds", time.time() - init_time)
self.force_change = False
self._next_live_priority_check = 0.0 # monotonic timestamp for throttled live priority checks
# All sports and content managers now handled via plugins
logger.info("All sports and content managers now handled via plugin system")
@@ -396,20 +425,64 @@ class DisplayController:
# Set up live priority checker
self.vegas_coordinator.set_live_priority_checker(self._check_live_priority)
# Set up interrupt checker for on-demand/wifi status
# Set up interrupt checker for on-demand/wifi status and follower mode
def _vegas_interrupt():
return self._check_vegas_interrupt() or self.sync_manager.is_follower_active()
self.vegas_coordinator.set_interrupt_checker(
self._check_vegas_interrupt,
_vegas_interrupt,
check_interval=10 # Check every 10 frames (~80ms at 125 FPS)
)
# Set up plugin update tick to keep data fresh during Vegas mode
self.vegas_coordinator.set_update_tick(
self._tick_plugin_updates_for_vegas,
interval=1.0
)
# Run plugin updates inside the Vegas loop so the inter-iteration
# gap is <1 ms (nothing left for _tick_plugin_updates() to do).
self.vegas_coordinator.set_update_callback(self._tick_plugin_updates)
# Wire multi-display sync into Vegas render pipeline
follower_pos = self.config.get("sync", {}).get("follower_position", "left")
self.vegas_coordinator.set_sync_manager(self.sync_manager, follower_pos)
logger.info("Vegas mode coordinator initialized")
# Follower does NOT build its own initial scroll image — the leader
# pushes its image via TCP as soon as set_on_follower_connected fires.
# A local build would create a different (wrong) image that could
# temporarily replace the leader's correct one.
# When the leader sends its scroll image (TCP), update our
# cached_array so both Pis have pixel-identical images.
import numpy as _np
def _on_leader_scroll_image(image):
vc = getattr(self, 'vegas_coordinator', None)
if vc and vc.render_pipeline:
rp = vc.render_pipeline
arr = _np.asarray(image.convert("RGB"), dtype=_np.uint8)
rp.scroll_helper.cached_image = image
rp.scroll_helper.cached_array = arr
rp.scroll_helper.total_scroll_width = image.width
self._follower_pending_new_image = False
logger.info(
"Sync: follower adopted leader scroll image %dx%d",
image.width, image.height,
)
self.sync_manager.set_on_scroll_image(_on_leader_scroll_image)
if self.sync_manager.role == SyncRole.LEADER:
# When a follower first connects, push the current scroll image so
# the follower doesn't have to wait for the next new_cycle event.
# Polls until the image is ready (Vegas may still be composing on startup).
def _on_follower_connected():
import time as _t
for _ in range(300): # up to 30s
vc = getattr(self, 'vegas_coordinator', None)
if vc and vc.render_pipeline:
img = vc.render_pipeline.scroll_helper.cached_image
if img is not None:
self.sync_manager.send_scroll_image(img)
return
_t.sleep(0.1)
logger.warning("Sync: no scroll image available to push to new follower")
self.sync_manager.set_on_follower_connected(_on_follower_connected)
except Exception as e:
logger.error("Failed to initialize Vegas mode: %s", e, exc_info=True)
self.vegas_coordinator = None
@@ -444,51 +517,16 @@ class DisplayController:
return False
def _tick_plugin_updates_for_vegas(self):
"""
Run scheduled plugin updates and return IDs of plugins that were updated.
Called periodically by the Vegas coordinator to keep plugin data fresh
during Vegas mode. Returns a list of plugin IDs whose data changed so
Vegas can refresh their content in the scroll.
Returns:
List of updated plugin IDs, or None if no updates occurred
"""
if not self.plugin_manager or not hasattr(self.plugin_manager, 'plugin_last_update'):
self._tick_plugin_updates()
return None
# Snapshot update timestamps before ticking
old_times = dict(self.plugin_manager.plugin_last_update)
# Run the scheduled updates
self._tick_plugin_updates()
# Detect which plugins were actually updated
updated = []
for plugin_id, new_time in self.plugin_manager.plugin_last_update.items():
if new_time > old_times.get(plugin_id, 0.0):
updated.append(plugin_id)
if updated:
logger.info("Vegas update tick: %d plugin(s) updated: %s", len(updated), updated)
return updated or None
def _check_schedule(self):
"""Check if display should be active based on schedule."""
# Get fresh config from config_service to support hot-reload
current_config = self.config_service.get_config()
schedule_config = current_config.get('schedule', {})
schedule_config = self.config.get('schedule', {})
# If schedule config doesn't exist or is empty, default to always active
if not schedule_config:
self.is_display_active = True
self._was_display_active = True # Track previous state for schedule change detection
return
# Check if schedule is explicitly disabled
# Default to True (schedule enabled) if 'enabled' key is missing for backward compatibility
if 'enabled' in schedule_config and not schedule_config.get('enabled', True):
@@ -498,7 +536,7 @@ class DisplayController:
return
# Get configured timezone, default to UTC
timezone_str = current_config.get('timezone', 'UTC')
timezone_str = self.config.get('timezone', 'UTC')
try:
tz = pytz.timezone(timezone_str)
except pytz.UnknownTimeZoneError:
@@ -596,18 +634,15 @@ class DisplayController:
Target brightness level (dim_brightness if in dim period,
normal brightness otherwise)
"""
# Get fresh config from config_service to support hot-reload
current_config = self.config_service.get_config()
# Get normal brightness from config
normal_brightness = current_config.get('display', {}).get('hardware', {}).get('brightness', 90)
normal_brightness = self.config.get('display', {}).get('hardware', {}).get('brightness', 90)
# If display is OFF via schedule, don't process dim schedule
if not self.is_display_active:
self.is_dimmed = False
return normal_brightness
dim_config = current_config.get('dim_schedule', {})
dim_config = self.config.get('dim_schedule', {})
# If dim schedule doesn't exist or is disabled, use normal brightness
if not dim_config or not dim_config.get('enabled', False):
@@ -615,7 +650,7 @@ class DisplayController:
return normal_brightness
# Get configured timezone
timezone_str = current_config.get('timezone', 'UTC')
timezone_str = self.config.get('timezone', 'UTC')
try:
tz = pytz.timezone(timezone_str)
except pytz.UnknownTimeZoneError:
@@ -722,21 +757,83 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
logger.exception("Error running scheduled plugin updates")
def _tick_plugin_updates_throttled(self, min_interval: float = 0.0):
"""Throttled version of _tick_plugin_updates for high-FPS loops.
_FOLLOWER_SEND_INTERVAL = 1.0 / 90 # raw bytes are cheap; 90fps > follower render rate
Args:
min_interval: Minimum seconds between calls. When <= 0 the
call passes straight through to _tick_plugin_updates so
plugin-configured update_interval values are never capped.
def _follower_rebuild_scroll_image(self) -> None:
"""Follower: rebuild the local Vegas scroll image so both Pis render from
the same fresh plugin data. Called at startup (after Vegas initializes)
and each time the leader broadcasts a new-cycle signal. Runs in a daemon
thread so it never blocks the 60fps render loop.
"""
if min_interval <= 0:
self._tick_plugin_updates()
try:
vc = getattr(self, 'vegas_coordinator', None)
if not vc:
logger.warning("Sync: follower has no vegas_coordinator — cannot build scroll image")
return
rp = vc.render_pipeline
if not rp:
logger.warning("Sync: follower vegas_coordinator has no render_pipeline")
return
logger.info("Sync: follower starting scroll image rebuild")
ok = rp.start_new_cycle()
if ok and rp.scroll_helper.cached_image is not None:
logger.info(
"Sync: follower scroll image ready — %dx%d",
rp.scroll_helper.cached_image.width,
rp.scroll_helper.cached_image.height,
)
else:
logger.warning(
"Sync: follower scroll image rebuild FAILED (ok=%s, cached=%s)",
ok, rp.scroll_helper.cached_image is not None,
)
except Exception as exc:
logger.warning("Sync: follower scroll image rebuild error: %s", exc, exc_info=True)
def _send_follower_frame(self, plugin_instance) -> None:
"""Leader: generate and send the follower's portion of the current frame.
The follower is physically to the LEFT of the leader in a right-to-left
scrolling ticker, so it shows content at scroll_position - display_width
(content that already scrolled off the leader's left edge).
Set sync.follower_position = "right" in config to invert this.
"""
if not (self.sync_manager and self.sync_manager.role == SyncRole.LEADER):
return
# Throttle to ~90fps via _FOLLOWER_SEND_INTERVAL — raw RGB bytes, no encode/decode
now = time.time()
if now - self._last_plugin_tick_time >= min_interval:
self._last_plugin_tick_time = now
self._tick_plugin_updates()
if now - getattr(self, '_last_follower_send', 0) < self._FOLLOWER_SEND_INTERVAL:
return
self._last_follower_send = now
follower_frame = None
width = self.display_manager.width
sync_cfg = self.config.get("sync", {})
sign = -1 if sync_cfg.get("follower_position", "left") == "left" else 1
offset = sign * width
# 1. Explicit hook — plugin opted in with get_offset_frame()
try:
follower_frame = plugin_instance.get_offset_frame(offset)
except Exception:
pass
# 2. Auto-detect — plugin has a scroll_helper (standard pattern for all
# scroll plugins). Works with zero plugin code changes.
if follower_frame is None:
try:
scroll_h = getattr(plugin_instance, 'scroll_helper', None)
if scroll_h is not None:
follower_frame = scroll_h.get_portion_at(scroll_h.scroll_position + offset)
except Exception:
pass
# 3. Mirror fallback — static plugins (clock, weather) show same frame
if follower_frame is None:
follower_frame = self.display_manager.image
if follower_frame is not None:
self.sync_manager.send_frame(follower_frame)
def _sleep_with_plugin_updates(self, duration: float, tick_interval: float = 1.0):
"""Sleep while continuing to service plugin update schedules."""
@@ -1373,6 +1470,88 @@ class DisplayController:
# Plugins update on their own schedules - no forced sync updates needed
# Each plugin has its own update_interval and background services
# Multi-display sync: follower mode — render frames received from leader.
# Plugin update() threads still run (via _tick_plugin_updates above) so
# data is fresh when we return to standalone if the leader goes offline.
if self.sync_manager.is_follower_active():
# Dead-reckoning follower render:
# Advance local position at configured speed each tick; snap or
# gently correct toward received scroll_x to absorb UDP jitter.
_now_dr = time.perf_counter()
_dt = _now_dr - getattr(self, '_follower_dr_last_t', _now_dr)
self._follower_dr_last_t = _now_dr
vc = getattr(self, 'vegas_coordinator', None)
rp = vc.render_pipeline if (vc and vc.render_pipeline) else None
width = self.display_manager.width
# Advance local position at Vegas scroll speed (px/s → px/tick)
vegas_speed = (
self.config.get('display', {})
.get('vegas_scroll', {})
.get('scroll_speed', 75)
)
local_x = getattr(self, '_follower_local_x', None)
if local_x is None:
local_x = float(width) # safe start (past pre-roll guard)
local_x += vegas_speed * _dt
# Pull latest position from leader (may be None if no packet yet)
scroll_x = self.sync_manager.get_latest_scroll_x()
if scroll_x is not None:
diff = scroll_x - local_x
total_w = (
rp.scroll_helper.total_scroll_width
if rp and rp.scroll_helper.total_scroll_width
else width * 4
)
if abs(diff) > total_w * 0.5:
# Large jump → cycle reset, snap immediately
local_x = float(scroll_x)
self._follower_pending_new_image = True
elif abs(diff) > 10:
# Moderate drift → 20% correction per tick
local_x += diff * 0.20
else:
# Near → gentle 5% correction
local_x += diff * 0.05
self._follower_local_x = local_x
if rp and rp.scroll_helper.cached_image is not None:
sync_cfg = self.config.get("sync", {})
sign = -1 if sync_cfg.get("follower_position", "left") == "left" else 1
# Hold last frame until TCP image arrives after cycle reset
if not getattr(self, "_follower_pending_new_image", False):
if local_x >= width:
rp.scroll_helper.scroll_position = local_x + sign * width
frame = rp.scroll_helper.get_visible_portion()
if frame is not None:
self._follower_last_frame = frame
elif scroll_x is None:
# Fallback: pixel frame before first scroll_x arrives
frame = self.sync_manager.get_latest_frame()
if frame is not None:
self._follower_last_frame = frame
display_frame = getattr(self, '_follower_last_frame', None)
if display_frame is not None:
self.display_manager.image = display_frame
self.display_manager._sync_render_allowed = True
self.display_manager.update_display()
self.display_manager._sync_render_allowed = False
# Precision deadline timer — keeps render at exactly 60fps
_deadline = getattr(self, '_follower_deadline', None)
_now = time.perf_counter()
if _deadline is None or _now > _deadline + 0.1:
_deadline = _now
_deadline += 1.0 / 60
self._follower_deadline = _deadline
_sleep = _deadline - time.perf_counter()
if _sleep > 0:
time.sleep(_sleep)
continue
# Process any deferred updates that may have accumulated
# This also cleans up expired updates to prevent memory leaks
self.display_manager.process_deferred_updates()
@@ -1746,7 +1925,7 @@ class DisplayController:
)
target_duration = max_duration
start_time = time.monotonic()
start_time = time.time()
def _should_exit_dynamic(elapsed_time: float) -> bool:
if not dynamic_enabled:
@@ -1805,34 +1984,19 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
logger.exception("Error during display update")
# Multi-display sync: send follower frame after each render
self._send_follower_frame(manager_to_display)
time.sleep(display_interval)
self._tick_plugin_updates_throttled(min_interval=1.0)
self._tick_plugin_updates()
self._poll_on_demand_requests()
self._check_on_demand_expiration()
# Check for live priority every ~30s so live
# games can interrupt long display durations
elapsed = time.monotonic() - start_time
now = time.monotonic()
if not self.on_demand_active and now >= self._next_live_priority_check:
self._next_live_priority_check = now + 30.0
live_mode = self._check_live_priority()
if live_mode and live_mode != active_mode:
logger.info("Live priority detected during high-FPS loop: %s", live_mode)
self.current_display_mode = live_mode
self.force_change = True
try:
self.current_mode_index = self.available_modes.index(live_mode)
except ValueError:
pass
# continue the main while loop to skip
# post-loop rotation/sleep logic
break
if self.current_display_mode != active_mode:
logger.debug("Mode changed during high-FPS loop, breaking early")
break
elapsed = time.time() - start_time
if elapsed >= target_duration:
logger.debug(
"Reached high-FPS target duration %.2fs for mode %s",
@@ -1862,7 +2026,7 @@ class DisplayController:
time.sleep(display_interval)
self._tick_plugin_updates()
elapsed = time.monotonic() - start_time
elapsed = time.time() - start_time
if elapsed >= target_duration:
logger.debug(
"Reached standard target duration %.2fs for mode %s",
@@ -1889,25 +2053,11 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
logger.exception("Error during display update")
# Multi-display sync: send follower frame after each render
self._send_follower_frame(manager_to_display)
self._poll_on_demand_requests()
self._check_on_demand_expiration()
# Check for live priority every ~30s so live
# games can interrupt long display durations
now = time.monotonic()
if not self.on_demand_active and now >= self._next_live_priority_check:
self._next_live_priority_check = now + 30.0
live_mode = self._check_live_priority()
if live_mode and live_mode != active_mode:
logger.info("Live priority detected during display loop: %s", live_mode)
self.current_display_mode = live_mode
self.force_change = True
try:
self.current_mode_index = self.available_modes.index(live_mode)
except ValueError:
pass
break
if self.current_display_mode != active_mode:
logger.info("Mode changed during display loop from %s to %s, breaking early", active_mode, self.current_display_mode)
break
@@ -1921,26 +2071,19 @@ class DisplayController:
loop_completed = True
break
# If live priority preempted the display loop, skip
# all post-loop logic (remaining sleep, rotation) and
# restart the main loop so the live mode displays
# immediately.
if self.current_display_mode != active_mode:
continue
# Ensure we honour minimum duration when not dynamic and loop ended early
if (
not dynamic_enabled
and not loop_completed
and not needs_high_fps
):
elapsed = time.monotonic() - start_time
elapsed = time.time() - start_time
remaining_sleep = max(0.0, max_duration - elapsed)
if remaining_sleep > 0:
self._sleep_with_plugin_updates(remaining_sleep)
if dynamic_enabled:
elapsed_total = time.monotonic() - start_time
elapsed_total = time.time() - start_time
cycle_done = self._plugin_cycle_complete(manager_to_display)
# Log cycle completion status and metrics

274
src/vegas_mode/coordinator.py
View File

@@ -90,13 +90,11 @@ class VegasModeCoordinator:
self._interrupt_check: Optional[Callable[[], bool]] = None
self._interrupt_check_interval: int = 10 # Check every N frames
# Plugin update tick for keeping data fresh during Vegas mode
self._update_tick: Optional[Callable[[], Optional[List[str]]]] = None
self._update_tick_interval: float = 1.0 # Tick every 1 second
self._update_thread: Optional[threading.Thread] = None
self._update_results: Optional[List[str]] = None
self._update_results_lock = threading.Lock()
self._last_update_tick_time: float = 0.0
# Plugin update callback — fired from a background thread inside the loop
# so the main loop's _tick_plugin_updates() finds nothing due when Vegas
# returns, eliminating the inter-iteration frozen-frame gap.
self._update_callback: Optional[Callable[[], None]] = None
self._update_tick_running: bool = False
# Config update tracking
self._config_version = 0
@@ -139,6 +137,25 @@ class VegasModeCoordinator:
"""Check if Vegas mode is currently running."""
return self._is_active
def set_sync_manager(self, sync_manager, follower_position: str = "left") -> None:
"""
Attach a DisplaySyncManager so Vegas mode sends the follower's portion
of the ticker to the second display on every rendered frame.
Args:
sync_manager: DisplaySyncManager instance, or None to disable sync
follower_position: "left" (default) or "right" — physical position of
the follower display relative to the leader
"""
if self.render_pipeline:
# Don't expose a standalone (no-op) manager to the pipeline — treat it as None
if sync_manager is not None and hasattr(sync_manager, 'role'):
from src.common.sync_manager import SyncRole
if sync_manager.role == SyncRole.STANDALONE:
sync_manager = None
self.render_pipeline.sync_manager = sync_manager
self.render_pipeline.sync_follower_left = (follower_position == "left")
def set_live_priority_checker(self, checker: Callable[[], Optional[str]]) -> None:
"""
Set the callback for checking live priority content.
@@ -166,24 +183,19 @@ class VegasModeCoordinator:
self._interrupt_check = checker
self._interrupt_check_interval = max(1, check_interval)
def set_update_tick(
self,
callback: Callable[[], Optional[List[str]]],
interval: float = 1.0
) -> None:
def set_update_callback(self, callback: Callable[[], None]) -> None:
"""
Set the callback for periodic plugin update ticking during Vegas mode.
Set a callback for running plugin updates from inside the Vegas loop.
This keeps plugin data fresh while the Vegas render loop is running.
The callback should run scheduled plugin updates and return a list of
plugin IDs that were actually updated, or None/empty if no updates occurred.
Fired in a daemon background thread every ~4 s so plugin data stays
fresh without blocking the render loop. The main loop's
_tick_plugin_updates() then finds all intervals already satisfied and
returns immediately, collapsing the inter-iteration gap to <1 ms.
Args:
callback: Callable that returns list of updated plugin IDs or None
interval: Seconds between update tick calls (default 1.0)
callback: Callable with no arguments (typically _tick_plugin_updates)
"""
self._update_tick = callback
self._update_tick_interval = max(0.5, interval)
self._update_callback = callback
def start(self) -> bool:
"""
@@ -237,9 +249,6 @@ class VegasModeCoordinator:
self.stats['total_runtime_seconds'] += time.time() - self._start_time
self._start_time = None
# Wait for in-flight background update before tearing down state
self._drain_update_thread()
# Cleanup components
self.render_pipeline.reset()
self.stream_manager.reset()
@@ -335,83 +344,101 @@ class VegasModeCoordinator:
last_fps_log_time = start_time
fps_frame_count = 0
self._last_update_tick_time = start_time
logger.info("Starting Vegas iteration for %.1fs", duration)
try:
while True:
# Check for STATIC mode plugin that should pause scroll
static_plugin = self._check_static_plugin_trigger()
if static_plugin:
if not self._handle_static_pause(static_plugin):
# Static pause was interrupted
while True:
# Check for STATIC mode plugin that should pause scroll
static_plugin = self._check_static_plugin_trigger()
if static_plugin:
if not self._handle_static_pause(static_plugin):
# Static pause was interrupted
return False
# After static pause, skip this segment and continue
self.stream_manager.get_next_segment() # Consume the segment
continue
# Run frame
if not self.run_frame():
# Check why we stopped
with self._state_lock:
if self._should_stop:
return False
if self._is_paused:
# Paused for live priority - let caller handle
return False
# After static pause, skip this segment and continue
self.stream_manager.get_next_segment() # Consume the segment
continue
# Run frame
if not self.run_frame():
# Check why we stopped
with self._state_lock:
if self._should_stop:
return False
if self._is_paused:
# Paused for live priority - let caller handle
return False
# Sleep for frame interval
time.sleep(frame_interval)
# Sleep for frame interval
time.sleep(frame_interval)
# Increment frame count and check for interrupt periodically
frame_count += 1
fps_frame_count += 1
# Increment frame count and check for interrupt periodically
frame_count += 1
fps_frame_count += 1
# Periodic FPS logging
current_time = time.time()
if current_time - last_fps_log_time >= fps_log_interval:
fps = fps_frame_count / (current_time - last_fps_log_time)
logger.info(
"Vegas FPS: %.1f (target: %d, frames: %d)",
fps, self.vegas_config.target_fps, fps_frame_count
)
last_fps_log_time = current_time
fps_frame_count = 0
# Periodic FPS logging
current_time = time.time()
if current_time - last_fps_log_time >= fps_log_interval:
fps = fps_frame_count / (current_time - last_fps_log_time)
logger.info(
"Vegas FPS: %.1f (target: %d, frames: %d)",
fps, self.vegas_config.target_fps, fps_frame_count
)
last_fps_log_time = current_time
fps_frame_count = 0
if (self._interrupt_check and
frame_count % self._interrupt_check_interval == 0):
try:
if self._interrupt_check():
logger.debug(
"Vegas interrupted by callback after %d frames",
frame_count
)
return False
except Exception:
# Log but don't let interrupt check errors stop Vegas
logger.exception("Interrupt check failed")
# Periodic plugin update tick to keep data fresh (non-blocking)
self._drive_background_updates()
if (self._interrupt_check and
frame_count % self._interrupt_check_interval == 0):
# Fire plugin update tick in a background thread every ~4 s.
# Running it here (rather than only between iterations) means the
# main loop's _tick_plugin_updates() finds all intervals already
# satisfied on return, so the inter-iteration gap is <1 ms and the
# display never shows a frozen frame between iterations.
_UPDATE_TICK_FRAMES = max(1, int(self.vegas_config.target_fps * 4)) # every 4 s regardless of FPS
if (self._update_callback and
frame_count % _UPDATE_TICK_FRAMES == 0 and
not self._update_tick_running):
self._update_tick_running = True
def _run_tick(cb=self._update_callback):
try:
if self._interrupt_check():
logger.debug(
"Vegas interrupted by callback after %d frames",
frame_count
)
return False
except Exception:
# Log but don't let interrupt check errors stop Vegas
logger.exception("Interrupt check failed")
cb()
finally:
self._update_tick_running = False
threading.Thread(
target=_run_tick, daemon=True, name="vegas-plugin-tick"
).start()
# Check elapsed time
elapsed = time.time() - start_time
if elapsed >= duration:
break
# Check elapsed time
elapsed = time.time() - start_time
if elapsed >= duration:
break
# Check for cycle completion
if self.render_pipeline.is_cycle_complete():
break
# NOTE: do NOT break on is_cycle_complete() here.
# When multi-display sync is active, breaking exits run_iteration()
# which causes a 2-3s delay before start_new_cycle() is called on
# the next run_iteration(). During that gap the scroll advances into
# the pre-roll zone, then start_new_cycle() resets it — producing a
# second visible jump on the follower display ~2.5s after the first.
#
# Instead, run_frame() handles cycle completion directly (it calls
# start_new_cycle() in the very next frame, 8ms later), collapsing
# the two events into a single clean transition.
#
# Without sync, the iteration now runs to its full duration and may
# cycle content multiple times within one iteration — acceptable for
# a continuous ticker.
logger.info("Vegas iteration completed after %.1fs", time.time() - start_time)
return True
finally:
# Ensure background update thread finishes before the main loop
# resumes its own _tick_plugin_updates() calls, preventing concurrent
# run_scheduled_updates() execution.
self._drain_update_thread()
logger.info("Vegas iteration completed after %.1fs", time.time() - start_time)
return True
def _check_live_priority(self) -> bool:
"""
@@ -500,71 +527,6 @@ class VegasModeCoordinator:
if self._pending_config is None:
self._pending_config_update = False
def _run_update_tick_background(self) -> None:
"""Run the plugin update tick in a background thread.
Stores results for the render loop to pick up on its next iteration,
so the scroll never blocks on API calls.
"""
try:
updated_plugins = self._update_tick()
if updated_plugins:
with self._update_results_lock:
# Accumulate rather than replace to avoid losing notifications
# if a previous result hasn't been picked up yet
if self._update_results is None:
self._update_results = updated_plugins
else:
self._update_results.extend(updated_plugins)
except Exception:
logger.exception("Background plugin update tick failed")
def _drain_update_thread(self, timeout: float = 2.0) -> None:
"""Wait for any in-flight background update thread to finish.
Called when transitioning out of Vegas mode so the main-loop
``_tick_plugin_updates`` call doesn't race with a still-running
background thread.
"""
if self._update_thread is not None and self._update_thread.is_alive():
self._update_thread.join(timeout=timeout)
if self._update_thread.is_alive():
logger.warning(
"Background update thread did not finish within %.1fs", timeout
)
def _drive_background_updates(self) -> None:
"""Collect finished background update results and launch new ticks.
Safe to call from both the main render loop and the static-pause
wait loop so that plugin data stays fresh regardless of which
code path is active.
"""
# 1. Collect results from a previously completed background update
with self._update_results_lock:
ready_results = self._update_results
self._update_results = None
if ready_results:
for pid in ready_results:
self.mark_plugin_updated(pid)
# 2. Kick off a new background update if interval elapsed and none running
current_time = time.time()
if (self._update_tick and
current_time - self._last_update_tick_time >= self._update_tick_interval):
thread_alive = (
self._update_thread is not None
and self._update_thread.is_alive()
)
if not thread_alive:
self._last_update_tick_time = current_time
self._update_thread = threading.Thread(
target=self._run_update_tick_background,
daemon=True,
name="vegas-update-tick",
)
self._update_thread.start()
def mark_plugin_updated(self, plugin_id: str) -> None:
"""
Notify that a plugin's data has been updated.
@@ -683,8 +645,10 @@ class VegasModeCoordinator:
logger.info("Static pause interrupted by live priority")
return False
# Keep plugin data fresh during static pause
self._drive_background_updates()
# Yield immediately if multi-display follower mode becomes active
if self._interrupt_check and self._interrupt_check():
logger.info("Static pause interrupted by sync follower mode")
return False
# Sleep in small increments to remain responsive
time.sleep(0.1)

116
src/vegas_mode/render_pipeline.py
View File

@@ -52,6 +52,10 @@ class RenderPipeline:
self.config = config
self.display_manager = display_manager
self.stream_manager = stream_manager
self.sync_manager = None # Optional DisplaySyncManager — set by coordinator
self.sync_follower_left = True # True = follower is LEFT of leader (default)
self._sync_send_interval = 1.0 / 90 # raw bytes are cheap; 90fps > follower render rate
self._last_sync_send = 0.0
# Display dimensions (handle both property and method access patterns)
self.display_width = (
@@ -208,13 +212,14 @@ class RenderPipeline:
# total_distance_scrolled >= total_scroll_width + display_width.
# That extra display_width of travel causes a "wrap-around" phase
# where scroll_position resets to ~0 and the first plugin's content
# re-enters from the right — the user sees this 2-3 s window as
# "a plugin partially displaying before the next one starts."
# re-enters from the right — the user sees this 2-3 s of re-entry
# as "a plugin partially displaying before the next one starts."
#
# We end the cycle as soon as total_distance_scrolled reaches
# total_scroll_width (the wrap-around point), before any second-pass
# content becomes visible. scroll_helper.is_scroll_complete() is
# kept as a fallback for edge-cases where that threshold is skipped.
# content becomes visible. The scroll_helper's own is_scroll_complete()
# check is kept as a fallback for any edge-cases where that threshold
# is never hit.
at_wrap_point = (
not self._cycle_complete and
self.scroll_helper.total_distance_scrolled >= self.scroll_helper.total_scroll_width
@@ -228,19 +233,16 @@ class RenderPipeline:
"Scroll cycle complete after %.1fs",
time.time() - self._cycle_start_time
)
# Push blank immediately so the hardware never shows
# post-wrap content while the coordinator recomposes.
# Push blank immediately so the hardware never shows any
# post-wrap content while the coordinator recomposes the
# next cycle (~100 ms).
try:
from PIL import Image as _Image
blank = _Image.new('RGB', (self.display_width, self.display_height))
self.display_manager.image = blank
self.display_manager.update_display()
except (ImportError, OSError, RuntimeError, ValueError, TypeError, MemoryError) as exc:
logger.error(
"Failed to push blank frame at cycle end "
"(display=%dx%d): %s",
self.display_width, self.display_height, exc
)
except Exception:
logger.exception("Failed to write blank frame to display at cycle end")
return True # Cycle done; coordinator starts new cycle next frame
# Get visible portion
@@ -252,6 +254,15 @@ class RenderPipeline:
self.display_manager.image = visible_frame
self.display_manager.update_display()
# Multi-display sync: send scroll position to follower.
# The follower renders from its own cached_array (kept identical to the
# leader's via TCP image transfer at each new_cycle) at scroll_x ± display_width.
if self.sync_manager:
now = time.time()
if now - self._last_sync_send >= self._sync_send_interval:
self._last_sync_send = now
self.sync_manager.send_scroll_x(self.scroll_helper.scroll_position)
# Update scrolling state
self.display_manager.set_scrolling_state(True)
@@ -291,33 +302,38 @@ class RenderPipeline:
if self._cycle_complete:
return True
# When multi-display sync is active, defer mid-cycle hot swaps until the
# cycle ends naturally. Hot swaps block the render loop for 15-30ms while
# the image is rebuilt, causing a freeze+jump that the follower perceives
# as a speed-up. Deferring to cycle boundaries keeps transitions clean.
# Staging buffer content is still pre-loaded; it just applies at cycle end.
if self.sync_manager is not None:
return False
# Check if we need more content in the buffer
buffer_status = self.stream_manager.get_buffer_status()
if buffer_status['staging_count'] > 0:
return True
# Trigger recompose when pending updates affect visible segments
if self.stream_manager.has_pending_updates_for_visible_segments():
return True
return False
def hot_swap_content(self) -> bool:
"""
Hot-swap to new composed content.
Called when staging buffer has updated content or pending updates exist.
Preserves scroll position for mid-cycle updates to prevent visual jumps.
Called when staging buffer has updated content.
Swaps atomically to prevent visual glitches.
Returns:
True if swap occurred
"""
try:
# Save scroll position for mid-cycle updates
saved_position = self.scroll_helper.scroll_position
saved_total_distance = self.scroll_helper.total_distance_scrolled
saved_total_width = max(1, self.scroll_helper.total_scroll_width)
was_mid_cycle = not self._cycle_complete
# Snapshot position before swap so we can reposition after.
# The new image has completely different content — if scroll_position
# is left unchanged it lands at an arbitrary mid-content point in the
# new image, causing a visible jump on both displays.
old_width = self.scroll_helper.total_scroll_width
old_pos = self.scroll_helper.scroll_position
# Process any pending updates
self.stream_manager.process_updates()
@@ -325,20 +341,24 @@ class RenderPipeline:
# Recompose with updated content
if self.compose_scroll_content():
# Map scroll position proportionally into the new image width so
# we resume at the same relative progress through the content.
# This keeps the visual tempo consistent and avoids the jump that
# occurred when old scroll_position landed arbitrarily in new image.
new_width = self.scroll_helper.total_scroll_width
if old_width > 0 and new_width > 0:
ratio = (old_pos % old_width) / old_width
self.scroll_helper.scroll_position = ratio * new_width
else:
self.scroll_helper.scroll_position = 0.0
self.stats['hot_swaps'] += 1
# Restore scroll position for mid-cycle updates so the
# scroll continues from where it was instead of jumping to 0
if was_mid_cycle:
new_total_width = max(1, self.scroll_helper.total_scroll_width)
progress_ratio = min(saved_total_distance / saved_total_width, 0.999)
self.scroll_helper.total_distance_scrolled = progress_ratio * new_total_width
self.scroll_helper.scroll_position = min(
saved_position,
float(new_total_width - 1)
)
self.scroll_helper.scroll_complete = False
self._cycle_complete = False
logger.debug("Hot-swap completed (mid_cycle_restore=%s)", was_mid_cycle)
logger.debug(
"Hot-swap completed: scroll repositioned %.0f→%.0f (%.1f%% of new %dpx image)",
old_pos, self.scroll_helper.scroll_position,
(self.scroll_helper.scroll_position / new_width * 100) if new_width else 0,
new_width,
)
return True
return False
@@ -373,7 +393,29 @@ class RenderPipeline:
return False
# Compose new scroll content
return self.compose_scroll_content()
result = self.compose_scroll_content()
if result and self.sync_manager:
# When sync is active, start the leader at display_width instead of 0.
# This skips the initial black gap so the leader immediately shows content.
# The follower starts at position 0 (the gap) which looks like a clean
# blank transition rather than near-end content wrapping around.
self.scroll_helper.scroll_position = float(self.display_width)
if result and self.sync_manager:
# Signal follower that a new cycle started (triggers its own rebuild)
self.sync_manager.send_new_cycle()
# Push the actual scroll image over TCP so follower has identical pixels.
# Done in a background thread to not block the render loop (~15ms transfer).
if self.scroll_helper.cached_image is not None:
import threading as _t
_t.Thread(
target=self.sync_manager.send_scroll_image,
args=(self.scroll_helper.cached_image,),
daemon=True, name="sync-image-push"
).start()
return result
def get_current_scroll_info(self) -> Dict[str, Any]:
"""Get current scroll state information."""

3260
web_interface/blueprints/api_v3.py
View File

File diff suppressed because it is too large Load Diff

173
web_interface/templates/v3/partials/display.html
View File

@@ -49,9 +49,9 @@
name="chain_length"
value="{{ main_config.display.hardware.chain_length or 2 }}"
min="1"
max="32"
max="8"
class="form-control">
<p class="mt-1 text-sm text-gray-600">Number of LED panels chained together (e.g. 2 for 128×32, 5 for 320×32)</p>
<p class="mt-1 text-sm text-gray-600">Number of LED panels chained together</p>
</div>
<div class="form-group">
@@ -386,6 +386,68 @@
</div>
</div>
<!-- Multi-Display Sync Settings -->
<div class="bg-gray-50 rounded-lg p-4 mt-6">
<div class="flex items-center justify-between mb-4">
<div>
<h3 class="text-md font-medium text-gray-900">
<i class="fas fa-clone mr-2"></i>Multi-Display Sync
</h3>
<p class="mt-1 text-sm text-gray-600">
Extend scrolling content across two LED matrix display units over WiFi.
Both displays must have identical rows and cols. Chain length may differ.
</p>
</div>
</div>
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
<div class="form-group">
<label for="sync_role" class="block text-sm font-medium text-gray-700">Role</label>
<select id="sync_role" name="sync_role" class="form-control" onchange="updateSyncUI()">
<option value="standalone" {% if main_config.get('sync', {}).get('role', 'standalone') == 'standalone' %}selected{% endif %}>Standalone (disabled)</option>
<option value="leader" {% if main_config.get('sync', {}).get('role', 'standalone') == 'leader' %}selected{% endif %}>Leader (drives scroll)</option>
<option value="follower" {% if main_config.get('sync', {}).get('role', 'standalone') == 'follower' %}selected{% endif %}>Follower (receives frames)</option>
</select>
<p class="mt-1 text-sm text-gray-600">Set Leader on one Pi, Follower on the other. Restart required after changing.</p>
</div>
<div class="form-group">
<label for="sync_port" class="block text-sm font-medium text-gray-700">UDP Port</label>
<input type="number"
id="sync_port"
name="sync_port"
value="{{ main_config.get('sync', {}).get('port', 5765) }}"
min="1024"
max="65535"
class="form-control">
<p class="mt-1 text-sm text-gray-600">
Must match on both Pis. If ufw is active:
<code class="text-xs bg-gray-200 px-1 rounded">sudo ufw allow {{ main_config.get('sync', {}).get('port', 5765) }}/udp</code>
</p>
</div>
<div class="form-group" id="sync_position_group" style="display:none">
<label for="sync_follower_position" class="block text-sm font-medium text-gray-700">Position</label>
<select id="sync_follower_position" name="sync_follower_position" class="form-control">
<option value="left" {% if main_config.get('sync', {}).get('follower_position', 'left') == 'left' %}selected{% endif %}>Left of leader</option>
<option value="right" {% if main_config.get('sync', {}).get('follower_position', 'left') == 'right' %}selected{% endif %}>Right of leader</option>
</select>
<p class="mt-1 text-sm text-gray-600">Which side of the leader display this unit sits on.</p>
</div>
</div>
<!-- Live status indicator (populated by JS) -->
<div id="sync_status_bar" class="mt-4 hidden">
<div id="sync_status_content" class="flex items-start space-x-2 p-3 rounded-lg border text-sm"></div>
</div>
<!-- Incompatibility detail (shown when error) -->
<div id="sync_error_detail" class="mt-2 hidden">
<p class="text-xs text-yellow-700 bg-yellow-50 border border-yellow-200 rounded p-2" id="sync_error_text"></p>
<p class="text-xs text-gray-500 mt-1">rows and cols must match between displays. chain_length may differ.</p>
</div>
</div>
<!-- Submit Button -->
<div class="flex justify-end">
<button type="submit"
@@ -643,4 +705,111 @@ if (typeof window.fixInvalidNumberInputs !== 'function') {
initPluginOrderList();
}
})();
// Multi-Display Sync UI
(function() {
function updateSyncUI() {
const role = document.getElementById('sync_role').value;
const bar = document.getElementById('sync_status_bar');
const posGroup = document.getElementById('sync_position_group');
if (role === 'standalone') {
bar.classList.add('hidden');
document.getElementById('sync_error_detail').classList.add('hidden');
posGroup.style.display = 'none';
} else {
bar.classList.remove('hidden');
posGroup.style.display = role === 'follower' ? '' : 'none';
pollSyncStatus();
}
}
window.updateSyncUI = updateSyncUI;
function pollSyncStatus() {
const role = document.getElementById('sync_role') && document.getElementById('sync_role').value;
if (!role || role === 'standalone') return;
fetch('/api/v3/sync/status')
.then(r => r.json())
.then(resp => {
const d = resp.data || {};
renderSyncStatus(d);
})
.catch(() => {
renderSyncStatus({state: 'unknown'});
});
}
function renderSyncStatus(d) {
const content = document.getElementById('sync_status_content');
const errorDetail = document.getElementById('sync_error_detail');
const errorText = document.getElementById('sync_error_text');
if (!content) return;
const state = d.state || 'unknown';
const role = d.role || 'unknown';
let icon, colorClass, text;
if (state === 'connected' || state === 'follower') {
icon = '●';
colorClass = 'bg-green-50 border-green-200 text-green-800';
const peer = d.peer_ip || d.leader_ip || 'peer';
text = role === 'leader'
? `Follower connected — ${peer} (chain ${d.peer_chain || '?'})`
: `Receiving from leader — ${peer}`;
errorDetail.classList.add('hidden');
} else if (state === 'incompatible') {
icon = '⚠';
colorClass = 'bg-yellow-50 border-yellow-200 text-yellow-800';
text = `Follower connected but incompatible panels`;
if (d.error) {
errorText.textContent = d.error;
errorDetail.classList.remove('hidden');
}
} else if (state === 'no_peer' || state === 'standalone') {
icon = '○';
colorClass = 'bg-gray-50 border-gray-200 text-gray-600';
text = role === 'leader' ? 'No follower detected' : 'Searching for leader…';
errorDetail.classList.add('hidden');
} else if (state === 'starting') {
icon = '○';
colorClass = 'bg-gray-50 border-gray-200 text-gray-500';
text = 'Display process starting…';
errorDetail.classList.add('hidden');
} else {
icon = '✕';
colorClass = 'bg-red-50 border-red-200 text-red-700';
text = 'Sync status unavailable';
errorDetail.classList.add('hidden');
}
content.className = `flex items-start space-x-2 p-3 rounded-lg border text-sm ${colorClass}`;
content.textContent = '';
const iconSpan = document.createElement('span');
iconSpan.className = 'font-bold text-lg leading-none';
iconSpan.textContent = icon;
const textSpan = document.createElement('span');
textSpan.textContent = text;
content.appendChild(iconSpan);
content.appendChild(textSpan);
}
// Initial UI state and polling — guard against duplicate intervals on re-run
function startSyncPolling() {
updateSyncUI();
if (!window.syncStatusInterval) {
window.syncStatusInterval = setInterval(pollSyncStatus, 5000);
}
}
if (document.readyState === 'loading') {
document.addEventListener('DOMContentLoaded', startSyncPolling);
} else {
startSyncPolling();
}
})();
</script>