mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-16 11:59:13 +00:00
feat(ipc)!: remove the cache-key mailboxes, control socket stage 5 (for the release after 3.8.1) (#773)
* feat(ipc)!: remove the cache-key mailboxes (control socket stage 5) The control socket is now the only way the web interface sends the display a command. The display stops reading display_on_demand_request and plugin_error_clear_request, and the web interface stops writing them. - Display: no mailbox poll (MailboxWatch, the 1 s / 0.25 s cadence, _consume_on_demand_request, the deprecation log) and no persisted display_on_demand_processed_id guard; the error publisher reads no clear request. CacheManager.file_signature and MailboxWatch are removed. - A write to either retired key is dropped by CacheManager.save_cache and logged once per writer, naming the plugin from the call stack (or the request's plugin_id), with the API to move to. - Web: on-demand start with no display listening starts the service (when start_service) and sends the request again once the socket answers (45 s, 10 s for a running service without a socket yet); every other failure is a 503 (400 for invalid_args). Stop answers 503 when no display listens, unless stop_service. errors/clear answers 503 with a reason-specific message instead of writing a request; clear_pending is always false. src.ipc.client.should_fall_back is replaced by display_not_listening. - Kept: display_current_state, display_on_demand_state, plugin_runtime_snapshot and the heartbeat (read whenever the socket cannot answer), and display_on_demand_config (the display's resume record). Tests: mailbox-only tests removed (test_on_demand_mailbox.py, the mailbox cadence, file_signature and MailboxWatch tests); tests that injected requests through the mailbox now use the socket queue or a plugin's in-process request. The run-loop harness sends on-demand requests over its fake control socket, so four golden traces change: on-demand starts and stops land at the request instant instead of the next 0.25 s mailbox look (one frame fewer on the screen they end), and in vegas.json within one frame instead of 263 ms, which shifts the later 1 s-throttled WiFi-notice check by under a second. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: the on-demand routes' other callers ack over the socket, not the mailbox test_api_v3_bool_coercion, test_api_v3_lazy_plugin_discovery and test_web_api's stop test checked the mailbox write; they now get an ack from a mocked control socket. The stop route touches no manager any more, so it leaves the removed-catch-all sample. A request that was sent is never 'not listening' (pins the sent check the mutation run let through). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web): a start that waits for the display answers 202 and is delivered in the background The start route held a request open for up to 45 s while a cold-started display loaded its plugins; the MQTT bridge (15 s timeout) and browsers reported a failure for a request that was then delivered. Now, when no display is listening, the route starts the service if asked and answers 202 with status "starting" at once. A single worker in the web process (web_interface/on_demand_dispatch.py) sends the request until the display acknowledges it or the wait runs out (45 s cold start, 10 s for a running service without a socket yet). A newer start supersedes the pending one; a stop cancels it (and succeeds, with cancelled_request_id, even with no display listening). The outcome is reported by /display/on-demand/status (source "web": starting, or error with start-timeout or the socket's reason, until the display publishes something newer) and by /display/current-status as on_demand_pending. Callers: the web UI's on-demand modal and "Preview on display" treat "starting" as taken (an info toast); the MQTT bridge already treats any non-error 2xx as success (now pinned by a test). Tests: the dispatcher (ack, retry then ack, start-timeout, other failures, superseded, an in-flight ack for a superseded start, stop while pending, a per-start wait, outcome lifetime); the routes (202, status routes while pending and after a timeout, a later display state replacing the failure, stop while pending, a new start superseding); a JS suite for app.js. Mutation check: 20 mutants on the worker, the routes and app.js, 20 killed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): a cancelled on-demand start never reaches the display after its replacement CI caught a race in test_a_new_start_supersedes_the_pending_one: the dispatcher's worker could read the old start, the route cancel it and send the new one, and the worker's send of the old one land after it. cancel() now waits out a send already in flight (the worker holds a send lock while it reads the pending start and sends it), so whatever the caller sends next lands after it. Pinned by test_a_cancel_waits_for_a_send_in_flight, which fails without the wait. The Linux-only TestRealSocket test for a display that went away now expects the 202 and the start-timeout that follows, as the route answers since the background dispatcher. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): a delivered on-demand start reads as starting until the display acts on it On ledpi (three cold starts) the display acknowledged the start as its socket opened, then took ~5 s to act on it while Vegas built its first strip; the status routes meanwhile showed the display's own idle state, so a UI polling every 700 ms flashed idle. The display's on-demand state now names the request it answers (request_id). A delivered start keeps reading as status "starting" with delivered: true, in /display/on-demand/status and as on_demand_pending in /display/current-status, until the display publishes state for that request id (a display without the field: any state newer than the delivery), for at most DELIVERED_SHOWN_SECONDS (30 s). The display's startup state, which can be published after the acknowledgement, names no request and does not end it. Tests: stays starting against the startup idle state (no id, an older id); the matching active state and the matching error take over; an older display's newer state takes over; the 30 s cap; the display's state names its request. Mutation check: 11 mutants, 11 killed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(display): on_demand_request_id has a class default for controllers built without __init__ _on_demand_state now publishes it, and test_state_stream_readers builds controllers with __new__. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: the case-spelling on-demand test patches _send_on_demand now the mailbox is gone Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * docs: blank line before 3.8.7 heading --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
@@ -5,18 +5,20 @@ the web UI and the MQTT bridge send) as "restart": with the service running it
|
||||
ran ``systemctl stop``, slept 1.5s and started it again. Every on-demand or
|
||||
"Preview on display" click therefore cold-restarted the display process --
|
||||
every plugin reloaded, the panel blank for seconds -- to deliver a request the
|
||||
running process polls for every ON_DEMAND_POLL_INTERVAL anyway (see
|
||||
test_on_demand_mailbox.py and test_display_pending_changes.py for the display
|
||||
side: the mailbox is read mid-dwell, mid-screen and mid-Vegas-iteration).
|
||||
running process takes within a frame over its control socket anyway (see
|
||||
test_display_pending_changes.py for the display side: commands land
|
||||
mid-dwell, mid-screen and mid-Vegas-iteration).
|
||||
|
||||
The restart did not buy anything either: a freshly started display restores
|
||||
only the on-demand session it saved itself (``display_on_demand_config``), so
|
||||
the new request reached it through the same mailbox, one cold start later.
|
||||
only the on-demand session it saved itself (``display_on_demand_config``).
|
||||
|
||||
This file previously pinned that restart path (it guarded a broken
|
||||
``import _pkg.time`` inside it). The path is gone; these tests pin its
|
||||
replacement: a running service is left alone, a stopped one is started (only
|
||||
when start_service is set), and the request lands in the mailbox either way.
|
||||
when start_service is set), and the request goes over the control socket
|
||||
either way -- to a stopped display once it has started and its socket is up,
|
||||
sent by the web process's dispatcher after the route has answered 202.
|
||||
Nothing is ever written to the cache: the file mailbox is gone (stage 5).
|
||||
|
||||
The service helpers are patched where they run. display.py binds
|
||||
_get_display_service_status by value, while _ensure_display_service_running
|
||||
@@ -25,6 +27,7 @@ _run_systemctl_command is the one place a systemctl command is issued.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
@@ -34,6 +37,8 @@ sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
|
||||
|
||||
from src.ipc import client as control_client # noqa: E402
|
||||
|
||||
START_URL = "/api/v3/display/on-demand/start"
|
||||
STOP_URL = "/api/v3/display/on-demand/stop"
|
||||
MAILBOX = "display_on_demand_request"
|
||||
@@ -46,7 +51,10 @@ def service(api_v3_module):
|
||||
|
||||
plugin_manager and config_manager are None so the route skips plugin
|
||||
resolution (not what is under test here). The cache is the blueprint's
|
||||
MagicMock cache_manager, so mailbox writes are visible as set() calls.
|
||||
MagicMock cache_manager, so a write would be visible as a set() call.
|
||||
|
||||
The control socket answers while the service is active and is missing
|
||||
(``no_socket``) while it is not; ``sent`` records what it carried.
|
||||
"""
|
||||
api_v3_module.api_v3.plugin_catalog = None
|
||||
api_v3_module.api_v3.config_manager = None
|
||||
@@ -62,7 +70,32 @@ def service(api_v3_module):
|
||||
state["active"] = False
|
||||
return {"returncode": 0, "stdout": "", "stderr": ""}
|
||||
|
||||
with patch("web_interface.blueprints.api_v3._get_display_service_status",
|
||||
sent = []
|
||||
|
||||
def socket(action):
|
||||
def call(request_id, *args, **kwargs):
|
||||
if not state["active"]:
|
||||
raise control_client.ControlError("no_socket", "x", sent=False)
|
||||
sent.append((action, request_id) + args)
|
||||
return {"accepted": True, "request_id": request_id}
|
||||
return call
|
||||
|
||||
class Clock:
|
||||
now = 1000.0
|
||||
|
||||
def monotonic(self):
|
||||
return self.now
|
||||
|
||||
def time(self):
|
||||
return self.now
|
||||
|
||||
def sleep(self, seconds):
|
||||
self.now += seconds
|
||||
|
||||
with patch("web_interface.blueprints.api_v3.time", Clock()), \
|
||||
patch(f"{DISPLAY}.control_client.on_demand_start", side_effect=socket("start")), \
|
||||
patch(f"{DISPLAY}.control_client.on_demand_stop", side_effect=socket("stop")), \
|
||||
patch("web_interface.blueprints.api_v3._get_display_service_status",
|
||||
side_effect=status), \
|
||||
patch("web_interface.blueprints.api_v3.display._get_display_service_status",
|
||||
side_effect=status), \
|
||||
@@ -74,6 +107,7 @@ def service(api_v3_module):
|
||||
"systemctl": run_systemctl,
|
||||
"stop_service": stop_service,
|
||||
"cache": api_v3_module.api_v3.cache_manager,
|
||||
"sent": sent,
|
||||
}
|
||||
|
||||
|
||||
@@ -99,19 +133,15 @@ class TestStartWhileTheServiceIsRunning:
|
||||
assert _systemctl_verbs(service["systemctl"]) == [], (
|
||||
"a running display service was sent a systemctl command")
|
||||
|
||||
def test_the_request_is_posted_for_the_running_display(self, api_v3_client, service):
|
||||
def test_the_request_is_sent_to_the_running_display(self, api_v3_client, service):
|
||||
response = api_v3_client.post(
|
||||
START_URL, json={"plugin_id": "weather", "mode": "weather_current",
|
||||
"duration": 60, "pinned": True})
|
||||
data = response.get_json()["data"]
|
||||
writes = _mailbox_writes(service["cache"])
|
||||
assert len(writes) == 1
|
||||
assert writes[0]["action"] == "start"
|
||||
assert writes[0]["request_id"] == data["request_id"]
|
||||
assert writes[0]["plugin_id"] == "weather"
|
||||
assert writes[0]["mode"] == "weather_current"
|
||||
assert writes[0]["duration"] == 60
|
||||
assert writes[0]["pinned"] is True
|
||||
assert service["sent"] == [("start", data["request_id"], "weather",
|
||||
"weather_current", 60, True)]
|
||||
assert data["transport"] == "socket"
|
||||
assert _mailbox_writes(service["cache"]) == []
|
||||
|
||||
def test_the_response_reports_the_service_was_not_started(self, api_v3_client, service):
|
||||
data = api_v3_client.post(START_URL, json={"plugin_id": "weather"}).get_json()["data"]
|
||||
@@ -129,11 +159,20 @@ class TestStartWhileTheServiceIsStopped:
|
||||
def test_start_service_starts_it_once_and_never_stops_it(self, api_v3_client, service):
|
||||
service["state"]["active"] = False
|
||||
response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
|
||||
assert response.status_code == 200, response.get_json()
|
||||
# Answered at once: the web process sends the request in the
|
||||
# background once the started display listens.
|
||||
assert response.status_code == 202, response.get_json()
|
||||
assert response.get_json()["status"] == "starting"
|
||||
assert _systemctl_verbs(service["systemctl"]) == ["start"]
|
||||
service["stop_service"].assert_not_called()
|
||||
# Written before the start, so the new process finds it on its first poll.
|
||||
assert len(_mailbox_writes(service["cache"])) == 1
|
||||
from web_interface import on_demand_dispatch
|
||||
dispatcher = on_demand_dispatch.current()
|
||||
deadline = time.monotonic() + 5
|
||||
while dispatcher.pending() and time.monotonic() < deadline:
|
||||
time.sleep(0.01)
|
||||
assert dispatcher.status()["status"] == "delivered"
|
||||
assert [s[0] for s in service["sent"]] == ["start"]
|
||||
assert _mailbox_writes(service["cache"]) == []
|
||||
|
||||
def test_without_start_service_it_is_left_stopped(self, api_v3_client, service):
|
||||
service["state"]["active"] = False
|
||||
@@ -141,6 +180,7 @@ class TestStartWhileTheServiceIsStopped:
|
||||
START_URL, json={"plugin_id": "weather", "start_service": "false"})
|
||||
assert response.status_code == 400
|
||||
assert _systemctl_verbs(service["systemctl"]) == []
|
||||
assert service["sent"] == []
|
||||
|
||||
def test_a_start_that_fails_is_reported(self, api_v3_client, service):
|
||||
service["state"]["active"] = False
|
||||
@@ -151,31 +191,12 @@ class TestStartWhileTheServiceIsStopped:
|
||||
assert response.get_json()["status"] == "error"
|
||||
|
||||
|
||||
class _Mailbox:
|
||||
"""The CacheManager calls the routes make, over a dict."""
|
||||
|
||||
def __init__(self):
|
||||
self.entries = {}
|
||||
|
||||
def set(self, key, value, ttl=None):
|
||||
self.entries[key] = value
|
||||
|
||||
def get(self, key, max_age=300, memory_ttl=None):
|
||||
return self.entries.get(key)
|
||||
|
||||
def delete(self, key):
|
||||
self.entries.pop(key, None)
|
||||
|
||||
|
||||
class TestARefusedStartLeavesNoRequestBehind:
|
||||
"""A start the route answers with an error must not run later.
|
||||
|
||||
The request was posted (to the mailbox, with the display stopped) before
|
||||
the route refused it, and the display reads the mailbox for an hour
|
||||
without looking at a request's age. So "Display service is not running"
|
||||
(start_service off) or "Failed to start display service" left the
|
||||
request waiting, and the next time the display started -- minutes later,
|
||||
by hand -- it ran that plugin, pinned if the request said so.
|
||||
With the mailbox, the request was posted before the route refused it, and
|
||||
a display started later ran it. Now nothing is written anywhere: the
|
||||
request only ever goes over the socket, to a display that answers.
|
||||
|
||||
A socket acknowledgement is the other side of it: the display answered,
|
||||
so it is running and has the request, whatever systemd says (a display
|
||||
@@ -184,76 +205,48 @@ class TestARefusedStartLeavesNoRequestBehind:
|
||||
"""
|
||||
|
||||
@pytest.fixture
|
||||
def mailbox(self, api_v3_module, service):
|
||||
box = _Mailbox()
|
||||
api_v3_module.api_v3.cache_manager = box
|
||||
def stopped(self, service):
|
||||
service["state"]["active"] = False
|
||||
return box
|
||||
return service
|
||||
|
||||
@pytest.mark.parametrize("body", [
|
||||
{"plugin_id": "weather", "start_service": False},
|
||||
{"plugin_id": "weather"}, # start_service defaults on
|
||||
])
|
||||
def test_a_socket_ack_is_a_success_whatever_systemd_says(
|
||||
self, api_v3_client, service, mailbox, body):
|
||||
self, api_v3_client, stopped, body):
|
||||
with patch(f"{DISPLAY}.control_client.on_demand_start",
|
||||
side_effect=lambda request_id, *a: {"accepted": True}):
|
||||
response = api_v3_client.post(START_URL, json=body)
|
||||
assert response.status_code == 200, response.get_json()
|
||||
assert response.get_json()["data"]["transport"] == "socket"
|
||||
assert MAILBOX not in mailbox.entries
|
||||
assert _systemctl_verbs(service["systemctl"]) == [], (
|
||||
assert _systemctl_verbs(stopped["systemctl"]) == [], (
|
||||
"a unit was started beside a display that answered the socket")
|
||||
|
||||
def test_without_start_service_the_request_is_taken_back(
|
||||
self, api_v3_client, service, mailbox):
|
||||
def test_without_start_service_nothing_is_left_behind(self, api_v3_client, stopped):
|
||||
response = api_v3_client.post(START_URL, json={
|
||||
"plugin_id": "weather", "pinned": True, "start_service": False})
|
||||
assert response.status_code == 400
|
||||
assert response.get_json()["status"] == "error"
|
||||
assert MAILBOX not in mailbox.entries
|
||||
assert stopped["cache"].set.call_count == 0
|
||||
assert stopped["sent"] == []
|
||||
|
||||
def test_a_start_that_fails_takes_its_request_back(self, api_v3_client, service, mailbox):
|
||||
service["systemctl"].side_effect = lambda args: {
|
||||
def test_a_start_that_fails_leaves_nothing_behind(self, api_v3_client, stopped):
|
||||
stopped["systemctl"].side_effect = lambda args: {
|
||||
"returncode": 1, "stdout": "", "stderr": "denied"}
|
||||
response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
|
||||
assert response.status_code == 500
|
||||
assert MAILBOX not in mailbox.entries
|
||||
|
||||
def test_a_newer_request_is_left_alone_on_the_400(self, api_v3_client, service, mailbox):
|
||||
newer = {"request_id": "someone-else", "action": "start", "plugin_id": "clock"}
|
||||
|
||||
def stopped_and_another_post_lands(*args):
|
||||
mailbox.entries[MAILBOX] = newer
|
||||
return {"active": False}
|
||||
|
||||
with patch(f"{DISPLAY}._get_display_service_status",
|
||||
side_effect=stopped_and_another_post_lands):
|
||||
response = api_v3_client.post(START_URL, json={
|
||||
"plugin_id": "weather", "start_service": False})
|
||||
assert response.status_code == 400
|
||||
assert mailbox.entries[MAILBOX] is newer
|
||||
|
||||
def test_a_newer_request_is_left_alone_on_the_500(self, api_v3_client, service, mailbox):
|
||||
newer = {"request_id": "someone-else", "action": "start", "plugin_id": "clock"}
|
||||
|
||||
def start_fails_after_another_post(args):
|
||||
mailbox.entries[MAILBOX] = newer
|
||||
return {"returncode": 1, "stdout": "", "stderr": "denied"}
|
||||
|
||||
service["systemctl"].side_effect = start_fails_after_another_post
|
||||
response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
|
||||
assert response.status_code == 500
|
||||
assert mailbox.entries[MAILBOX] is newer
|
||||
assert stopped["cache"].set.call_count == 0
|
||||
assert stopped["sent"] == []
|
||||
|
||||
|
||||
class TestStop:
|
||||
def test_stop_posts_a_stop_request_and_leaves_the_service_running(
|
||||
def test_stop_sends_a_stop_request_and_leaves_the_service_running(
|
||||
self, api_v3_client, service):
|
||||
response = api_v3_client.post(STOP_URL, json={})
|
||||
assert response.status_code == 200, response.get_json()
|
||||
writes = _mailbox_writes(service["cache"])
|
||||
assert [w["action"] for w in writes] == ["stop"]
|
||||
assert [s[0] for s in service["sent"]] == ["stop"]
|
||||
assert _mailbox_writes(service["cache"]) == []
|
||||
service["stop_service"].assert_not_called()
|
||||
assert _systemctl_verbs(service["systemctl"]) == []
|
||||
|
||||
|
||||
Reference in New Issue
Block a user