rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-16 03:49:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(ipc)!: remove the cache-key mailboxes, control socket stage 5 (for the release after 3.8.1) (#773)

Browse Source 
* feat(ipc)!: remove the cache-key mailboxes (control socket stage 5)

The control socket is now the only way the web interface sends the display a
command. The display stops reading display_on_demand_request and
plugin_error_clear_request, and the web interface stops writing them.

- Display: no mailbox poll (MailboxWatch, the 1 s / 0.25 s cadence,
  _consume_on_demand_request, the deprecation log) and no persisted
  display_on_demand_processed_id guard; the error publisher reads no clear
  request. CacheManager.file_signature and MailboxWatch are removed.
- A write to either retired key is dropped by CacheManager.save_cache and
  logged once per writer, naming the plugin from the call stack (or the
  request's plugin_id), with the API to move to.
- Web: on-demand start with no display listening starts the service (when
  start_service) and sends the request again once the socket answers (45 s,
  10 s for a running service without a socket yet); every other failure is
  a 503 (400 for invalid_args). Stop answers 503 when no display listens,
  unless stop_service. errors/clear answers 503 with a reason-specific
  message instead of writing a request; clear_pending is always false.
  src.ipc.client.should_fall_back is replaced by display_not_listening.
- Kept: display_current_state, display_on_demand_state,
  plugin_runtime_snapshot and the heartbeat (read whenever the socket cannot
  answer), and display_on_demand_config (the display's resume record).

Tests: mailbox-only tests removed (test_on_demand_mailbox.py, the mailbox
cadence, file_signature and MailboxWatch tests); tests that injected
requests through the mailbox now use the socket queue or a plugin's
in-process request. The run-loop harness sends on-demand requests over its
fake control socket, so four golden traces change: on-demand starts and
stops land at the request instant instead of the next 0.25 s mailbox look
(one frame fewer on the screen they end), and in vegas.json within one
frame instead of 263 ms, which shifts the later 1 s-throttled WiFi-notice
check by under a second.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: the on-demand routes' other callers ack over the socket, not the mailbox

test_api_v3_bool_coercion, test_api_v3_lazy_plugin_discovery and
test_web_api's stop test checked the mailbox write; they now get an ack
from a mocked control socket. The stop route touches no manager any more,
so it leaves the removed-catch-all sample. A request that was sent is
never 'not listening' (pins the sent check the mutation run let through).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web): a start that waits for the display answers 202 and is delivered in the background

The start route held a request open for up to 45 s while a cold-started
display loaded its plugins; the MQTT bridge (15 s timeout) and browsers
reported a failure for a request that was then delivered.

Now, when no display is listening, the route starts the service if asked
and answers 202 with status "starting" at once. A single worker in the web
process (web_interface/on_demand_dispatch.py) sends the request until the
display acknowledges it or the wait runs out (45 s cold start, 10 s for a
running service without a socket yet). A newer start supersedes the
pending one; a stop cancels it (and succeeds, with cancelled_request_id,
even with no display listening). The outcome is reported by
/display/on-demand/status (source "web": starting, or error with
start-timeout or the socket's reason, until the display publishes
something newer) and by /display/current-status as on_demand_pending.

Callers: the web UI's on-demand modal and "Preview on display" treat
"starting" as taken (an info toast); the MQTT bridge already treats any
non-error 2xx as success (now pinned by a test).

Tests: the dispatcher (ack, retry then ack, start-timeout, other failures,
superseded, an in-flight ack for a superseded start, stop while pending,
a per-start wait, outcome lifetime); the routes (202, status routes while
pending and after a timeout, a later display state replacing the failure,
stop while pending, a new start superseding); a JS suite for app.js.
Mutation check: 20 mutants on the worker, the routes and app.js, 20 killed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): a cancelled on-demand start never reaches the display after its replacement

CI caught a race in test_a_new_start_supersedes_the_pending_one: the
dispatcher's worker could read the old start, the route cancel it and send
the new one, and the worker's send of the old one land after it. cancel()
now waits out a send already in flight (the worker holds a send lock while
it reads the pending start and sends it), so whatever the caller sends next
lands after it. Pinned by test_a_cancel_waits_for_a_send_in_flight, which
fails without the wait.

The Linux-only TestRealSocket test for a display that went away now
expects the 202 and the start-timeout that follows, as the route answers
since the background dispatcher.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): a delivered on-demand start reads as starting until the display acts on it

On ledpi (three cold starts) the display acknowledged the start as its
socket opened, then took ~5 s to act on it while Vegas built its first
strip; the status routes meanwhile showed the display's own idle state, so
a UI polling every 700 ms flashed idle.

The display's on-demand state now names the request it answers
(request_id). A delivered start keeps reading as status "starting" with
delivered: true, in /display/on-demand/status and as on_demand_pending in
/display/current-status, until the display publishes state for that
request id (a display without the field: any state newer than the
delivery), for at most DELIVERED_SHOWN_SECONDS (30 s). The display's
startup state, which can be published after the acknowledgement, names no
request and does not end it.

Tests: stays starting against the startup idle state (no id, an older id);
the matching active state and the matching error take over; an older
display's newer state takes over; the 30 s cap; the display's state names
its request. Mutation check: 11 mutants, 11 killed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(display): on_demand_request_id has a class default for controllers built without __init__

_on_demand_state now publishes it, and test_state_stream_readers builds
controllers with __new__.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: the case-spelling on-demand test patches _send_on_demand now the mailbox is gone

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* docs: blank line before 3.8.7 heading

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-10-15 17:31:35 -04:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 456a1ac16c
commit 397333ae59
46 changed files with 2312 additions and 1788 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/_run_loop_harness.py
+15 -9
View File
@@ -155,13 +155,6 @@ class FakeCache:
self._writes += 1
self._written[key] = self._writes
def file_signature(self, key):
"""CacheManager.file_signature: None without a file, else a value
that changes with every write."""
if key not in self.data:
return None
return (self._written.get(key, 0), 0, 0)
def delete(self, key):
self.data.pop(key, None)
@@ -729,10 +722,23 @@ class RunLoopHarness:
self.controller.available_modes.append(mode)
def on_demand_request(self, t: float, request_id: str, action: str = "start", **fields):
"""An on-demand start or stop from the web interface at ``t``: a
command on the control socket (served for the run if no test did),
which is the only way the web interface reaches the display."""
from src.ipc.contract import Command, parse_args
from src.ipc.server import QueuedCommand
server = self.controller._control_server
if not isinstance(server, FakeControlServer):
server = self.control_socket()
cmd = Command.ON_DEMAND_START if action == "start" else Command.ON_DEMAND_STOP
command = QueuedCommand(request_id=request_id, cmd=cmd,
args=parse_args(cmd, fields if action == "start" else {}),
received_at=0.0)
def post():
self.log("request", f"{action}:{request_id}")
self.cache.set("display_on_demand_request",
{"request_id": request_id, "action": action, **fields})
server.queue.append(command)
self.clock.at(t, post)
def restore_on_demand(self, plugin_id: str, mode: Optional[str] = None,
test/conftest.py
+14
View File
@@ -328,6 +328,20 @@ def _hermetic_control_socket(monkeypatch):
monkeypatch.setenv(SOCKET_PATH_ENV, 'off')
@pytest.fixture(autouse=True)
def _no_pending_on_demand_dispatch():
"""Drop the web process's on-demand dispatcher after each test, so a
start one test left pending is not still being sent in the next."""
yield
module = sys.modules.get('web_interface.on_demand_dispatch')
if module is None:
return
dispatcher = module.current()
if dispatcher is not None:
dispatcher.cancel('test-teardown')
module.reset_for_tests()
@pytest.fixture(autouse=True)
def _hermetic_unit_refresh(monkeypatch, tmp_path_factory):
"""Keep updates' systemd unit refresh off the host.
test/fixtures/run_loop_golden/on_demand.json
Vendored
+3 -3
View File
@@ -1,16 +1,16 @@
{
"screens": [
[0.0, "clock", 20.0, "duration", 20, false],
[20.0, "weather", 5.0, "on-demand-start", 6, true],
[20.0, "weather", 5.0, "on-demand-start", 5, true],
[25.0, "sports_recent", 15.0, "duration", 15, true],
[40.0, "sports_upcoming", 15.0, "duration", 15, true],
[55.0, "sports_recent", 15.0, "duration", 15, true],
[70.0, "sports_upcoming", 15.0, "duration", 15, true],
[85.0, "sports_recent", 10.0, "on-demand-requested-stop", 11, true],
[85.0, "sports_recent", 10.0, "on-demand-requested-stop", 10, true],
[95.0, "weather", 20.0, "duration", 20, true],
[115.0, "sports_recent", 15.0, "duration", 15, true],
[130.0, "sports_upcoming", 15.0, "duration", 15, true],
[145.0, "clock", 5.0, "on-demand-start", 6, true],
[145.0, "clock", 5.0, "on-demand-start", 5, true],
[150.0, "weather", 20.0, "duration", 20, true],
[170.0, "weather", 10.0, "on-demand-expired", 10, true],
[180.0, "clock", 20.0, "duration", 20, true],
test/fixtures/run_loop_golden/on_demand_named_live.json
Vendored
+4 -4
View File
@@ -1,18 +1,18 @@
{
"screens": [
[0.0, "clock", 5.0, "on-demand-start", 6, false],
[0.0, "clock", 5.0, "on-demand-start", 5, false],
[5.0, "sports_live", 15.0, "duration", 15, true],
[20.0, "sports_recent", 15.0, "duration", 15, true],
[35.0, "sports_upcoming", 5.0, "on-demand-requested-stop", 6, true],
[35.0, "sports_upcoming", 5.0, "on-demand-requested-stop", 5, true],
[40.0, "clock", 20.0, "duration", 20, true],
[60.0, "sports_live", 15.0, "display-false", 11, true],
[75.0, "sports_recent", 15.0, "duration", 15, true],
[90.0, "sports_upcoming", 10.0, "on-demand-start", 11, true],
[90.0, "sports_upcoming", 10.0, "on-demand-start", 10, true],
[100.0, "sports_live", 0.0, "empty", 1, true],
[100.0, "sports_recent", 15.0, "duration", 15, true],
[115.0, "sports_upcoming", 15.0, "duration", 15, true],
[130.0, "sports_live", 0.0, "empty", 1, true],
[130.0, "sports_recent", 10.0, "on-demand-requested-stop", 11, true],
[130.0, "sports_recent", 10.0, "on-demand-requested-stop", 10, true],
[140.0, "sports_upcoming", 15.0, "duration", 15, true],
[155.0, "clock", 5.0, "horizon", 5, true]
],
test/fixtures/run_loop_golden/on_demand_pinned.json
Vendored
+2 -2
View File
@@ -1,11 +1,11 @@
{
"screens": [
[0.0, "clock", 12.0, "on-demand-start", 13, false],
[0.0, "clock", 12.0, "on-demand-start", 12, false],
[12.0, "sports_upcoming", 15.0, "duration", 15, true],
[27.0, "sports_upcoming", 15.0, "duration", 15, true],
[42.0, "sports_upcoming", 15.0, "duration", 15, true],
[57.0, "sports_upcoming", 15.0, "duration", 15, true],
[72.0, "sports_upcoming", 8.0, "on-demand-start", 9, true],
[72.0, "sports_upcoming", 8.0, "on-demand-start", 8, true],
[80.0, "app_a", 0.0, "empty", 1, true],
[80.0, "app_b", 10.0, "duration", 10, true],
[90.0, "app_a", 0.0, "empty", 1, true],
test/fixtures/run_loop_golden/vegas.json
Vendored
+11 -11
View File
@@ -6,22 +6,22 @@
[70.255, "sports_live", 20.0, "duration", 20, true],
[90.255, "sports_live", 20.0, "display-false", 11, false],
[110.255, "<vegas>", 30.008, "duration", 3751, null],
[140.263, "<vegas>", 10.0, "on-demand-start", 1250, null],
[150.263, "clock", 20.0, "duration", 20, true],
[170.263, "clock", 5.0, "on-demand-expired", 5, true],
[175.263, "<vegas>", 24.959, "vegas-interrupt", 3120, null],
[200.222, "<wifi>", 3.0, "duration", 6, null],
[203.222, "<vegas>", 30.008, "duration", 3751, null],
[233.23, "<vegas>", 26.77, "horizon", 3347, null]
[140.263, "<vegas>", 9.744, "on-demand-start", 1218, null],
[150.007, "clock", 20.0, "duration", 20, true],
[170.007, "clock", 5.0, "on-demand-expired", 5, true],
[175.007, "<vegas>", 25.999, "vegas-interrupt", 3250, null],
[201.006, "<wifi>", 2.0, "duration", 4, null],
[203.006, "<vegas>", 30.008, "duration", 3751, null],
[233.014, "<vegas>", 26.986, "horizon", 3374, null]
],
"events": [
[70.255, "vegas-live"],
[70.255, "live", "sports_live"],
[150.0, "request", "start:v1"],
[150.263, "on-demand-start", "clock"],
[150.263, "vegas-interrupt"],
[175.263, "on-demand-expired"],
[150.007, "on-demand-start", "clock"],
[150.007, "vegas-interrupt"],
[175.007, "on-demand-expired"],
[200.0, "wifi-file", "Connected to HomeNet"],
[200.222, "vegas-interrupt"]
[201.006, "vegas-interrupt"]
]
}
test/js/run_all.js
+1
View File
@@ -33,6 +33,7 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
'unit/test_overview_reconciliation_poll.js',
'unit/test_display_partial_ids.js',
'unit/test_general_web_login_token.js',
'unit/test_on_demand_starting.js',
'unit/test_general_display_autostart.js'];
const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
'dom/test_tools_sections.js', 'dom/test_cache_page.js',
test/js/unit/test_on_demand_starting.js
+69
View File
@@ -0,0 +1,69 @@
// POST /api/v3/display/on-demand/start answers 202 with status "starting"
// when the display service has to be started first: the request is taken,
// and the web process sends it once the display listens. "Preview on
// display" (app.js) must read that as taken -- an info toast and the
// floating preview opened -- not as a failure. Runs the shipped app.js in a
// vm with a minimal fake DOM, as test_restart_banner.js does.
const fs = require('fs');
const path = require('path');
const vm = require('vm');
const V3 = path.resolve(__dirname, '../../../web_interface/static/v3');
let pass = 0, fail = 0;
const ok = (label, cond, extra) => cond
? (pass++, console.log(' ok ' + label))
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
function load(answer) {
const notes = [];
const opened = [];
const noop = () => {};
const document = {
body: { addEventListener: noop },
addEventListener: noop,
getElementById: () => null,
querySelector: () => null,
querySelectorAll: () => [],
};
const window = { addEventListener: noop, getApp: () => null };
const context = {
window, document, console,
sessionStorage: { setItem: noop, removeItem: noop, getItem: () => null },
showNotification: (m, t) => notes.push([m, t]),
setTimeout: () => 0,
fetch: () => Promise.resolve({ json: () => Promise.resolve(answer) }),
};
vm.createContext(context);
vm.runInContext(fs.readFileSync(path.join(V3, 'app.js'), 'utf8'), context);
window.toggleFloatingPreview = (open) => opened.push(open);
return { window, notes, opened };
}
async function preview(answer) {
const t = load(answer);
t.window.previewPluginNow('weather');
for (let i = 0; i < 5; i++) await Promise.resolve();
return t;
}
(async () => {
console.log('\npreviewPluginNow');
{
const t = await preview({ status: 'starting', message: 'The display service is starting',
data: { request_id: 'r1', pending: true } });
ok('a 202 "starting" answer is an info toast, not an error',
t.notes.length === 1 && t.notes[0][1] === 'info', t.notes);
ok('and the preview opens', t.opened.length === 1 && t.opened[0] === true, t.opened);
}
{
const t = await preview({ status: 'success', data: { request_id: 'r1' } });
ok('a 200 success still opens it', t.opened.length === 1 && t.notes[0][1] === 'success', t.notes);
}
{
const t = await preview({ status: 'error', message: 'no display' });
ok('an error does not', t.opened.length === 0 && t.notes[0][1] === 'error', t.notes);
}
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})();
test/test_api_v3_bool_coercion.py
+4 -1
View File
@@ -54,7 +54,10 @@ class TestOnDemandStart:
def service(self, api_v3_module):
api_v3_module.api_v3.plugin_catalog = None
api_v3_module.api_v3.config_manager = None
with patch("web_interface.blueprints.api_v3.display._get_display_service_status",
# The display answers the control socket (the only way in).
with patch("web_interface.blueprints.api_v3.display.control_client.on_demand_start",
side_effect=lambda request_id, *a: {"accepted": True}), \
patch("web_interface.blueprints.api_v3.display._get_display_service_status",
return_value={"active": True}), \
patch("web_interface.blueprints.api_v3.display._stop_display_service") as stop, \
patch("web_interface.blueprints.api_v3.display._ensure_display_service_running",
test/test_api_v3_display_modes.py
+1 -2
View File
@@ -160,8 +160,7 @@ class TestOneBadConfigSectionDoesNotBlankTheList:
class TestOnDemandUsesTheRegisteredSpelling:
def test_a_mode_differing_in_case_is_sent_as_registered(self, client):
with patch('web_interface.blueprints.api_v3.display._deliver_on_demand',
return_value=('socket', None)) as deliver:
with patch('web_interface.blueprints.api_v3.display._send_on_demand') as deliver:
response = client.post('/api/v3/display/on-demand/start',
json={'plugin_id': 'football-scoreboard',
'mode': 'NFL_LIVE', 'start_service': False})
test/test_api_v3_lazy_plugin_discovery.py
+15 -4
View File
@@ -77,11 +77,22 @@ def fresh_web_process(api_v3_module, plugins_dir):
@pytest.fixture
def display_service(api_v3_module):
"""Keep on-demand start away from systemctl and the real cache."""
cache = api_v3_module.api_v3.cache_manager = MagicMock()
with patch('web_interface.blueprints.api_v3.display._get_display_service_status') as status:
"""Keep on-demand start away from systemctl and the real cache; the
display's control socket is a mock that acks. What it was sent is
recorded as ``.set(cmd, request)`` calls on the yielded mock."""
api_v3_module.api_v3.cache_manager = MagicMock()
sent = MagicMock()
def ack(request_id, plugin_id, mode, *a, **kw):
sent.set('on_demand.start', {'request_id': request_id,
'plugin_id': plugin_id, 'mode': mode})
return {'accepted': True}
with patch('web_interface.blueprints.api_v3.display._get_display_service_status') as status, \
patch('web_interface.blueprints.api_v3.display.control_client.on_demand_start',
side_effect=ack):
status.return_value = {'active': True}
yield cache
yield sent
def _start(client, **body):
test/test_api_v3_on_demand_restart.py
+77 -84
View File
@@ -5,18 +5,20 @@ the web UI and the MQTT bridge send) as "restart": with the service running it
ran ``systemctl stop``, slept 1.5s and started it again. Every on-demand or
"Preview on display" click therefore cold-restarted the display process --
every plugin reloaded, the panel blank for seconds -- to deliver a request the
running process polls for every ON_DEMAND_POLL_INTERVAL anyway (see
test_on_demand_mailbox.py and test_display_pending_changes.py for the display
side: the mailbox is read mid-dwell, mid-screen and mid-Vegas-iteration).
running process takes within a frame over its control socket anyway (see
test_display_pending_changes.py for the display side: commands land
mid-dwell, mid-screen and mid-Vegas-iteration).
The restart did not buy anything either: a freshly started display restores
only the on-demand session it saved itself (``display_on_demand_config``), so
the new request reached it through the same mailbox, one cold start later.
only the on-demand session it saved itself (``display_on_demand_config``).
This file previously pinned that restart path (it guarded a broken
``import _pkg.time`` inside it). The path is gone; these tests pin its
replacement: a running service is left alone, a stopped one is started (only
when start_service is set), and the request lands in the mailbox either way.
when start_service is set), and the request goes over the control socket
either way -- to a stopped display once it has started and its socket is up,
sent by the web process's dispatcher after the route has answered 202.
Nothing is ever written to the cache: the file mailbox is gone (stage 5).
The service helpers are patched where they run. display.py binds
_get_display_service_status by value, while _ensure_display_service_running
@@ -25,6 +27,7 @@ _run_systemctl_command is the one place a systemctl command is issued.
"""
import sys
import time
from pathlib import Path
from unittest.mock import patch
@@ -34,6 +37,8 @@ sys.path.insert(0, str(Path(__file__).parent.parent))
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
from src.ipc import client as control_client # noqa: E402
START_URL = "/api/v3/display/on-demand/start"
STOP_URL = "/api/v3/display/on-demand/stop"
MAILBOX = "display_on_demand_request"
@@ -46,7 +51,10 @@ def service(api_v3_module):
plugin_manager and config_manager are None so the route skips plugin
resolution (not what is under test here). The cache is the blueprint's
MagicMock cache_manager, so mailbox writes are visible as set() calls.
MagicMock cache_manager, so a write would be visible as a set() call.
The control socket answers while the service is active and is missing
(``no_socket``) while it is not; ``sent`` records what it carried.
"""
api_v3_module.api_v3.plugin_catalog = None
api_v3_module.api_v3.config_manager = None
@@ -62,7 +70,32 @@ def service(api_v3_module):
state["active"] = False
return {"returncode": 0, "stdout": "", "stderr": ""}
with patch("web_interface.blueprints.api_v3._get_display_service_status",
sent = []
def socket(action):
def call(request_id, *args, **kwargs):
if not state["active"]:
raise control_client.ControlError("no_socket", "x", sent=False)
sent.append((action, request_id) + args)
return {"accepted": True, "request_id": request_id}
return call
class Clock:
now = 1000.0
def monotonic(self):
return self.now
def time(self):
return self.now
def sleep(self, seconds):
self.now += seconds
with patch("web_interface.blueprints.api_v3.time", Clock()), \
patch(f"{DISPLAY}.control_client.on_demand_start", side_effect=socket("start")), \
patch(f"{DISPLAY}.control_client.on_demand_stop", side_effect=socket("stop")), \
patch("web_interface.blueprints.api_v3._get_display_service_status",
side_effect=status), \
patch("web_interface.blueprints.api_v3.display._get_display_service_status",
side_effect=status), \
@@ -74,6 +107,7 @@ def service(api_v3_module):
"systemctl": run_systemctl,
"stop_service": stop_service,
"cache": api_v3_module.api_v3.cache_manager,
"sent": sent,
}
@@ -99,19 +133,15 @@ class TestStartWhileTheServiceIsRunning:
assert _systemctl_verbs(service["systemctl"]) == [], (
"a running display service was sent a systemctl command")
def test_the_request_is_posted_for_the_running_display(self, api_v3_client, service):
def test_the_request_is_sent_to_the_running_display(self, api_v3_client, service):
response = api_v3_client.post(
START_URL, json={"plugin_id": "weather", "mode": "weather_current",
"duration": 60, "pinned": True})
data = response.get_json()["data"]
writes = _mailbox_writes(service["cache"])
assert len(writes) == 1
assert writes[0]["action"] == "start"
assert writes[0]["request_id"] == data["request_id"]
assert writes[0]["plugin_id"] == "weather"
assert writes[0]["mode"] == "weather_current"
assert writes[0]["duration"] == 60
assert writes[0]["pinned"] is True
assert service["sent"] == [("start", data["request_id"], "weather",
"weather_current", 60, True)]
assert data["transport"] == "socket"
assert _mailbox_writes(service["cache"]) == []
def test_the_response_reports_the_service_was_not_started(self, api_v3_client, service):
data = api_v3_client.post(START_URL, json={"plugin_id": "weather"}).get_json()["data"]
@@ -129,11 +159,20 @@ class TestStartWhileTheServiceIsStopped:
def test_start_service_starts_it_once_and_never_stops_it(self, api_v3_client, service):
service["state"]["active"] = False
response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
assert response.status_code == 200, response.get_json()
# Answered at once: the web process sends the request in the
# background once the started display listens.
assert response.status_code == 202, response.get_json()
assert response.get_json()["status"] == "starting"
assert _systemctl_verbs(service["systemctl"]) == ["start"]
service["stop_service"].assert_not_called()
# Written before the start, so the new process finds it on its first poll.
assert len(_mailbox_writes(service["cache"])) == 1
from web_interface import on_demand_dispatch
dispatcher = on_demand_dispatch.current()
deadline = time.monotonic() + 5
while dispatcher.pending() and time.monotonic() < deadline:
time.sleep(0.01)
assert dispatcher.status()["status"] == "delivered"
assert [s[0] for s in service["sent"]] == ["start"]
assert _mailbox_writes(service["cache"]) == []
def test_without_start_service_it_is_left_stopped(self, api_v3_client, service):
service["state"]["active"] = False
@@ -141,6 +180,7 @@ class TestStartWhileTheServiceIsStopped:
START_URL, json={"plugin_id": "weather", "start_service": "false"})
assert response.status_code == 400
assert _systemctl_verbs(service["systemctl"]) == []
assert service["sent"] == []
def test_a_start_that_fails_is_reported(self, api_v3_client, service):
service["state"]["active"] = False
@@ -151,31 +191,12 @@ class TestStartWhileTheServiceIsStopped:
assert response.get_json()["status"] == "error"
class _Mailbox:
"""The CacheManager calls the routes make, over a dict."""
def __init__(self):
self.entries = {}
def set(self, key, value, ttl=None):
self.entries[key] = value
def get(self, key, max_age=300, memory_ttl=None):
return self.entries.get(key)
def delete(self, key):
self.entries.pop(key, None)
class TestARefusedStartLeavesNoRequestBehind:
"""A start the route answers with an error must not run later.
The request was posted (to the mailbox, with the display stopped) before
the route refused it, and the display reads the mailbox for an hour
without looking at a request's age. So "Display service is not running"
(start_service off) or "Failed to start display service" left the
request waiting, and the next time the display started -- minutes later,
by hand -- it ran that plugin, pinned if the request said so.
With the mailbox, the request was posted before the route refused it, and
a display started later ran it. Now nothing is written anywhere: the
request only ever goes over the socket, to a display that answers.
A socket acknowledgement is the other side of it: the display answered,
so it is running and has the request, whatever systemd says (a display
@@ -184,76 +205,48 @@ class TestARefusedStartLeavesNoRequestBehind:
"""
@pytest.fixture
def mailbox(self, api_v3_module, service):
box = _Mailbox()
api_v3_module.api_v3.cache_manager = box
def stopped(self, service):
service["state"]["active"] = False
return box
return service
@pytest.mark.parametrize("body", [
{"plugin_id": "weather", "start_service": False},
{"plugin_id": "weather"}, # start_service defaults on
])
def test_a_socket_ack_is_a_success_whatever_systemd_says(
self, api_v3_client, service, mailbox, body):
self, api_v3_client, stopped, body):
with patch(f"{DISPLAY}.control_client.on_demand_start",
side_effect=lambda request_id, *a: {"accepted": True}):
response = api_v3_client.post(START_URL, json=body)
assert response.status_code == 200, response.get_json()
assert response.get_json()["data"]["transport"] == "socket"
assert MAILBOX not in mailbox.entries
assert _systemctl_verbs(service["systemctl"]) == [], (
assert _systemctl_verbs(stopped["systemctl"]) == [], (
"a unit was started beside a display that answered the socket")
def test_without_start_service_the_request_is_taken_back(
self, api_v3_client, service, mailbox):
def test_without_start_service_nothing_is_left_behind(self, api_v3_client, stopped):
response = api_v3_client.post(START_URL, json={
"plugin_id": "weather", "pinned": True, "start_service": False})
assert response.status_code == 400
assert response.get_json()["status"] == "error"
assert MAILBOX not in mailbox.entries
assert stopped["cache"].set.call_count == 0
assert stopped["sent"] == []
def test_a_start_that_fails_takes_its_request_back(self, api_v3_client, service, mailbox):
service["systemctl"].side_effect = lambda args: {
def test_a_start_that_fails_leaves_nothing_behind(self, api_v3_client, stopped):
stopped["systemctl"].side_effect = lambda args: {
"returncode": 1, "stdout": "", "stderr": "denied"}
response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
assert response.status_code == 500
assert MAILBOX not in mailbox.entries
def test_a_newer_request_is_left_alone_on_the_400(self, api_v3_client, service, mailbox):
newer = {"request_id": "someone-else", "action": "start", "plugin_id": "clock"}
def stopped_and_another_post_lands(*args):
mailbox.entries[MAILBOX] = newer
return {"active": False}
with patch(f"{DISPLAY}._get_display_service_status",
side_effect=stopped_and_another_post_lands):
response = api_v3_client.post(START_URL, json={
"plugin_id": "weather", "start_service": False})
assert response.status_code == 400
assert mailbox.entries[MAILBOX] is newer
def test_a_newer_request_is_left_alone_on_the_500(self, api_v3_client, service, mailbox):
newer = {"request_id": "someone-else", "action": "start", "plugin_id": "clock"}
def start_fails_after_another_post(args):
mailbox.entries[MAILBOX] = newer
return {"returncode": 1, "stdout": "", "stderr": "denied"}
service["systemctl"].side_effect = start_fails_after_another_post
response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
assert response.status_code == 500
assert mailbox.entries[MAILBOX] is newer
assert stopped["cache"].set.call_count == 0
assert stopped["sent"] == []
class TestStop:
def test_stop_posts_a_stop_request_and_leaves_the_service_running(
def test_stop_sends_a_stop_request_and_leaves_the_service_running(
self, api_v3_client, service):
response = api_v3_client.post(STOP_URL, json={})
assert response.status_code == 200, response.get_json()
writes = _mailbox_writes(service["cache"])
assert [w["action"] for w in writes] == ["stop"]
assert [s[0] for s in service["sent"]] == ["stop"]
assert _mailbox_writes(service["cache"]) == []
service["stop_service"].assert_not_called()
assert _systemctl_verbs(service["systemctl"]) == []
test/test_api_v3_on_demand_socket.py
+398 -85
View File
@@ -1,15 +1,12 @@
"""POST /display/on-demand/start and /stop: control socket first, mailbox fallback.
"""POST /display/on-demand/start and /stop: the control socket is the only way.
The routes hand the request to the display over the control socket
(src/ipc) and get an acknowledgement. When the socket could not carry the
request -- no socket (a stopped display, or one older than the socket), a
refused or timed-out connect, a display too old to know the command, a bug
in the client -- they write the file mailbox exactly as they did before the
socket existed. When the display had the request and failed it (a full
queue, bad arguments, no answer in time) the route says so and writes
nothing. These tests pin each path, that at most one of them is used, that
the response says which, and that the request id is the same either way (the
display deduplicates on it).
(src/ipc) and get an acknowledgement. The file mailbox they used to fall
back to (``display_on_demand_request``) is gone (stage 5), so nothing is
ever written to the cache. When no display is listening (a stopped one, or
one still starting) the start route starts the service if asked to and
sends the request again once the socket is up; every other failure is
answered as what it is.
The socket client is patched at the route's module attribute; the last class
runs a real server on a temp socket (Linux/macOS only).
@@ -17,6 +14,8 @@ runs a real server on a temp socket (Linux/macOS only).
import os
import sys
import time
import types
from pathlib import Path
from unittest.mock import patch
@@ -33,11 +32,12 @@ START_URL = "/api/v3/display/on-demand/start"
STOP_URL = "/api/v3/display/on-demand/stop"
MAILBOX = "display_on_demand_request"
CLIENT = "web_interface.blueprints.api_v3.display.control_client"
DISPLAY = "web_interface.blueprints.api_v3.display"
@pytest.fixture
def service(api_v3_module):
"""A running display service; records systemctl calls and mailbox writes."""
"""A running display service; records systemctl calls and cache writes."""
api_v3_module.api_v3.plugin_catalog = None
api_v3_module.api_v3.config_manager = None
state = {"active": True}
@@ -101,85 +101,398 @@ class TestSocketPath:
assert _mailbox_writes(service["cache"]) == []
class TestMailboxFallback:
@pytest.mark.parametrize("reason", [
"no_socket", "refused", "timeout", "closed", "bad_response", "invalid_request",
"busy", "forbidden", "unknown_command", "unsupported_version", "disabled",
"unsupported",
])
def test_a_request_the_socket_never_carried_writes_the_mailbox(
self, api_v3_client, service, reason):
# sent=False: the display never had it (no socket, a refused or
# timed-out connect, turned away at the door).
with patch(f"{CLIENT}.on_demand_start",
side_effect=control_client.ControlError(reason, "x", sent=False)):
resp = api_v3_client.post(START_URL, json={
"plugin_id": "weather", "mode": "weather_current",
"duration": 60, "pinned": True})
assert resp.status_code == 200
data = resp.get_json()["data"]
assert data["transport"] == "mailbox"
assert data["socket_error"] == reason
[write] = _mailbox_writes(service["cache"])
assert write["request_id"] == data["request_id"]
assert write["action"] == "start"
assert (write["plugin_id"], write["mode"], write["duration"], write["pinned"]) == \
("weather", "weather_current", 60, True)
class FakeTime:
"""Stands in for the route's ``time`` module: sleep() moves the clock."""
def test_a_client_bug_still_falls_back(self, api_v3_client, service):
def __init__(self):
self.now = 1000.0
self.sleeps = 0
def monotonic(self):
return self.now
def time(self):
return self.now
def sleep(self, seconds):
self.sleeps += 1
self.now += seconds
@pytest.fixture
def clock():
fake = FakeTime()
with patch("web_interface.blueprints.api_v3.time", fake):
yield fake
def _attempts(*outcomes, calls=None, clock=None):
"""A fake on_demand_start/stop that answers ``outcomes`` in turn (an
exception is raised, anything else acks); the last one repeats."""
outcomes = list(outcomes)
seen = calls if calls is not None else []
def attempt(request_id, *a, **kw):
seen.append(clock.now if clock is not None else None)
outcome = outcomes.pop(0) if len(outcomes) > 1 else outcomes[0]
if isinstance(outcome, BaseException):
raise outcome
return _ack(request_id)
return attempt
def _until(predicate, timeout=5.0):
end = time.monotonic() + timeout
while time.monotonic() < end:
if predicate():
return True
time.sleep(0.005)
return False
def _no_socket():
return control_client.ControlError("no_socket", "x", sent=False)
class TestNoDisplayListening:
"""No socket to talk to: the display is stopped, or still starting.
The route answers at once (202, ``status: "starting"``) and the web
process's dispatcher (web_interface/on_demand_dispatch.py) sends the
request until the display acknowledges it; the status routes report the
outcome. The dispatcher runs on real time with short waits here.
"""
@pytest.fixture(autouse=True)
def quick(self, monkeypatch, service):
from web_interface import on_demand_dispatch
monkeypatch.setattr(on_demand_dispatch, "RETRY_INTERVAL", 0.01)
monkeypatch.setattr(on_demand_dispatch, "START_WAIT_SECONDS", 2.0)
monkeypatch.setattr(f"{DISPLAY}.ON_DEMAND_SOCKET_WAIT_RUNNING_SECONDS", 1.0)
service["cache"].get.return_value = None # the display has published nothing
@staticmethod
def _outcome(status=None):
from web_interface import on_demand_dispatch
d = on_demand_dispatch.current()
assert d is not None
assert _until(lambda: not d.pending()), "the dispatcher never finished"
return d.status()
def test_a_display_still_starting_gets_the_request_once_it_listens(
self, api_v3_client, service):
calls = []
with patch(f"{CLIENT}.on_demand_start",
side_effect=_attempts(_no_socket(), _no_socket(), "ack", calls=calls)):
resp = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
assert resp.status_code == 202, resp.get_json()
body = resp.get_json()
assert body["status"] == "starting"
data = body["data"]
assert data["pending"] is True and data["socket_error"] == "no_socket"
assert data["service"]["started"] is False
outcome = self._outcome()
assert outcome["status"] == "delivered"
assert outcome["request_id"] == data["request_id"]
assert len(calls) == 3
assert not [c for c in service["calls"] if c[0] == "systemctl"]
assert _mailbox_writes(service["cache"]) == []
def test_a_stopped_display_is_started_and_answered_at_once(self, api_v3_client, service):
service["state"]["active"] = False
with patch(f"{CLIENT}.on_demand_start",
side_effect=_attempts(*[_no_socket()] * 6, "ack")) as start:
started = time.monotonic()
resp = api_v3_client.post(START_URL, json={"plugin_id": "weather",
"duration": 30})
answered = time.monotonic() - started
assert resp.status_code == 202, resp.get_json()
data = resp.get_json()["data"]
assert data["service"]["started"] is True
from web_interface import on_demand_dispatch
assert data["wait_seconds"] == on_demand_dispatch.START_WAIT_SECONDS
outcome = self._outcome()
assert answered < 1.0, "the route waited for the display"
assert service["calls"] == [("systemctl", "start")]
assert outcome["status"] == "delivered"
# The same request, the same id, every time.
assert {call.args[0] for call in start.call_args_list} == {data["request_id"]}
assert _mailbox_writes(service["cache"]) == []
def test_while_pending_the_status_routes_say_starting(self, api_v3_client, service):
service["state"]["active"] = False
with patch(f"{CLIENT}.on_demand_start", side_effect=_no_socket()):
rid = api_v3_client.post(START_URL, json={"plugin_id": "weather"}) \
.get_json()["data"]["request_id"]
status = api_v3_client.get("/api/v3/display/on-demand/status").get_json()["data"]
assert status["source"] == "web"
assert status["state"]["status"] == "starting"
assert status["state"]["request_id"] == rid
assert status["state"]["plugin_id"] == "weather"
current = api_v3_client.get("/api/v3/display/current-status").get_json()["data"]
assert current["on_demand_pending"]["status"] == "starting"
def test_a_display_that_never_comes_up_is_a_start_timeout(self, api_v3_client, service):
service["state"]["active"] = False
with patch(f"{CLIENT}.on_demand_start", side_effect=_no_socket()):
resp = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
assert resp.status_code == 202
outcome = self._outcome()
assert outcome["status"] == "error" and outcome["error"] == "start-timeout"
status = api_v3_client.get("/api/v3/display/on-demand/status").get_json()["data"]
assert status["state"]["status"] == "error"
assert status["state"]["error"] == "start-timeout"
current = api_v3_client.get("/api/v3/display/current-status").get_json()["data"]
assert current["on_demand_pending"]["error"] == "start-timeout"
def test_a_later_display_state_replaces_the_failure(self, api_v3_client, service):
service["state"]["active"] = False
with patch(f"{CLIENT}.on_demand_start", side_effect=_no_socket()):
api_v3_client.post(START_URL, json={"plugin_id": "weather"})
failed_at = self._outcome()["last_updated"]
later = {"active": True, "status": "active", "plugin_id": "clock",
"last_updated": failed_at + 5}
service["cache"].get.return_value = later
status = api_v3_client.get("/api/v3/display/on-demand/status").get_json()["data"]
assert status["state"]["plugin_id"] == "clock" and status["source"] == "cache"
def test_a_running_service_without_a_socket_is_waited_for_less(
self, api_v3_client, service):
with patch(f"{CLIENT}.on_demand_start", side_effect=_no_socket()):
resp = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
assert resp.status_code == 202
assert resp.get_json()["data"]["wait_seconds"] == 1.0
assert self._outcome()["error"] == "start-timeout"
assert not [c for c in service["calls"] if c[0] == "systemctl"]
def test_a_different_failure_while_waiting_ends_it(self, api_v3_client, service):
calls = []
busy = control_client.ControlError("busy", "x", sent=True)
with patch(f"{CLIENT}.on_demand_start",
side_effect=_attempts(_no_socket(), _no_socket(), busy, calls=calls)):
assert api_v3_client.post(START_URL, json={"plugin_id": "weather"}).status_code == 202
outcome = self._outcome()
assert outcome["status"] == "error" and outcome["error"] == "busy"
assert len(calls) == 3
def test_a_stop_while_pending_cancels_it(self, api_v3_client, service):
service["state"]["active"] = False
calls = []
with patch(f"{CLIENT}.on_demand_start",
side_effect=_attempts(_no_socket(), calls=calls)), \
patch(f"{CLIENT}.on_demand_stop", side_effect=_no_socket()):
rid = api_v3_client.post(START_URL, json={"plugin_id": "weather"}) \
.get_json()["data"]["request_id"]
resp = api_v3_client.post(STOP_URL, json={})
assert resp.status_code == 200, resp.get_json()
data = resp.get_json()["data"]
assert data["cancelled_request_id"] == rid
outcome = self._outcome()
n = len(calls)
time.sleep(0.05)
assert len(calls) == n, "the cancelled start was still being sent"
assert outcome["status"] == "idle" and outcome["last_event"] == "requested-stop"
status = api_v3_client.get("/api/v3/display/on-demand/status").get_json()["data"]
assert status["state"]["status"] == "idle" and status["source"] != "web"
# -- delivered, but the display has not acted on it yet ----------------
# The display acknowledges a start when its socket opens and acts on it
# seconds later (Vegas builds its first strip first: ~5 s on ledpi);
# meanwhile it publishes its own idle state, which must not flash.
def _delivered(self, api_v3_client, service):
with patch(f"{CLIENT}.on_demand_start",
side_effect=_attempts(_no_socket(), "ack")):
data = api_v3_client.post(START_URL, json={"plugin_id": "weather"}) .get_json()["data"]
outcome = self._outcome()
assert outcome["status"] == "delivered"
return data["request_id"], outcome["last_updated"]
@staticmethod
def _status(api_v3_client):
return api_v3_client.get("/api/v3/display/on-demand/status").get_json()["data"]
def test_a_delivered_start_stays_starting_until_the_display_answers(
self, api_v3_client, service):
rid, delivered_at = self._delivered(api_v3_client, service)
# The display's startup state: idle, published after the ack, for
# no request (or an older one).
for older in (None, "an-older-request"):
service["cache"].get.return_value = {
"active": False, "status": "idle", "request_id": older,
"last_updated": delivered_at + 1}
data = self._status(api_v3_client)
assert data["source"] == "web", older
assert data["state"]["status"] == "starting"
assert data["state"]["delivered"] is True
assert data["state"]["request_id"] == rid
current = api_v3_client.get("/api/v3/display/current-status").get_json()["data"]
assert current["on_demand_pending"]["delivered"] is True
def test_the_display_state_for_the_request_takes_over(self, api_v3_client, service):
rid, delivered_at = self._delivered(api_v3_client, service)
service["cache"].get.return_value = {
"active": True, "status": "active", "plugin_id": "weather",
"request_id": rid, "last_updated": delivered_at + 5}
data = self._status(api_v3_client)
assert data["source"] == "cache" and data["state"]["status"] == "active"
current = api_v3_client.get("/api/v3/display/current-status").get_json()["data"]
assert "on_demand_pending" not in current
def test_its_error_takes_over_too(self, api_v3_client, service):
rid, delivered_at = self._delivered(api_v3_client, service)
service["cache"].get.return_value = {
"active": False, "status": "error", "error": "load-failed",
"request_id": rid, "last_updated": delivered_at + 5}
assert self._status(api_v3_client)["state"]["error"] == "load-failed"
def test_an_older_display_answers_with_any_newer_state(self, api_v3_client, service):
# A display before request_id was published: timestamps decide.
_, delivered_at = self._delivered(api_v3_client, service)
service["cache"].get.return_value = {"active": False, "status": "idle",
"last_updated": delivered_at - 1}
assert self._status(api_v3_client)["source"] == "web"
service["cache"].get.return_value = {"active": True, "status": "active",
"last_updated": delivered_at + 1}
assert self._status(api_v3_client)["source"] == "cache"
def test_a_delivered_start_is_shown_for_a_limited_time(
self, api_v3_client, service, monkeypatch):
from web_interface import on_demand_dispatch
_, delivered_at = self._delivered(api_v3_client, service)
service["cache"].get.return_value = {"active": False, "status": "idle",
"request_id": None,
"last_updated": delivered_at + 1}
cap = on_demand_dispatch.DELIVERED_SHOWN_SECONDS
assert cap == 30.0
clock = types.SimpleNamespace(time=lambda: delivered_at + cap - 1,
monotonic=time.monotonic, sleep=time.sleep)
monkeypatch.setattr("web_interface.blueprints.api_v3.time", clock)
assert self._status(api_v3_client)["source"] == "web"
clock.time = lambda: delivered_at + cap + 1
data = self._status(api_v3_client)
assert data["source"] == "cache" and data["state"]["status"] == "idle"
current = api_v3_client.get("/api/v3/display/current-status").get_json()["data"]
assert "on_demand_pending" not in current
def test_a_new_start_supersedes_the_pending_one(self, api_v3_client, service):
service["state"]["active"] = False
with patch(f"{CLIENT}.on_demand_start", side_effect=_no_socket()):
old = api_v3_client.post(START_URL, json={"plugin_id": "weather"}) \
.get_json()["data"]["request_id"]
sent = []
service["state"]["active"] = True
with patch(f"{CLIENT}.on_demand_start",
side_effect=lambda rid, *a: sent.append(rid) or {"accepted": True}):
resp = api_v3_client.post(START_URL, json={"plugin_id": "clock"})
assert resp.status_code == 200
outcome = self._outcome()
# The old start is cancelled before the new one is sent, and
# cancel() waits out a send of it in flight: nothing of it lands
# after the new one.
new = resp.get_json()["data"]["request_id"]
assert sent[-1] == new and sent.count(new) == 1
assert outcome["status"] == "idle" and outcome["last_event"] == "superseded"
def test_a_service_that_will_not_start_is_an_error(self, api_v3_client, service, clock):
service["state"]["active"] = False
with patch("web_interface.blueprints.api_v3._run_systemctl_command",
return_value={"returncode": 1, "stdout": "", "stderr": "nope"}), \
patch(f"{CLIENT}.on_demand_start", side_effect=_no_socket()) as start:
resp = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
assert resp.status_code == 500
assert "Failed to start display service" in resp.get_json()["message"]
assert start.call_count == 1
def test_stop_with_no_display_running_is_an_error(self, api_v3_client, service, clock):
service["state"]["active"] = False
with patch(f"{CLIENT}.on_demand_stop", side_effect=_no_socket()) as stop:
resp = api_v3_client.post(STOP_URL, json={})
assert resp.status_code == 503
body = resp.get_json()
assert "not running" in body["message"]
assert body["data"]["socket_error"] == "no_socket"
assert stop.call_count == 1 and clock.sleeps == 0
assert service["calls"] == []
assert _mailbox_writes(service["cache"]) == []
def test_stop_with_a_running_service_but_no_socket_is_an_error(
self, api_v3_client, service, clock):
with patch(f"{CLIENT}.on_demand_stop",
side_effect=control_client.ControlError("refused", "x")):
resp = api_v3_client.post(STOP_URL, json={})
assert resp.status_code == 503
assert "still be starting" in resp.get_json()["message"]
def test_stop_with_stop_service_stops_it_anyway(self, api_v3_client, service, clock):
with patch(f"{CLIENT}.on_demand_stop", side_effect=_no_socket()), \
patch(f"{DISPLAY}._stop_display_service",
return_value={"active": False}) as stop:
resp = api_v3_client.post(STOP_URL, json={"stop_service": True})
assert resp.status_code == 200
assert resp.get_json()["data"]["socket_error"] == "no_socket"
stop.assert_called_once()
assert _mailbox_writes(service["cache"]) == []
def test_the_socket_is_off_in_the_test_suite(self, api_v3_client, service):
# conftest's _hermetic_control_socket: a suite run on a device must
# not drive the live display. Nothing is retried or started for it.
assert os.environ[c.SOCKET_PATH_ENV] == "off"
service["state"]["active"] = False
resp = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
assert resp.status_code == 503
assert resp.get_json()["data"]["socket_error"] in ("disabled", "unsupported")
assert service["calls"] == []
class TestNothingElseIsRetried:
@pytest.mark.parametrize("reason,sent,status", [
("timeout", False, 503), ("busy", False, 503), ("forbidden", False, 503),
("invalid_request", False, 503), ("disabled", False, 503),
("unsupported", False, 503), ("unknown_command", True, 503),
("unsupported_version", True, 503),
])
def test_answered_at_once_and_nothing_written(self, api_v3_client, service, clock,
reason, sent, status):
service["state"]["active"] = False
with patch(f"{CLIENT}.on_demand_start",
side_effect=control_client.ControlError(reason, "x", sent=sent)) as start:
resp = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
assert resp.status_code == status
body = resp.get_json()
assert body["status"] == "error"
assert body["data"]["socket_error"] == reason
assert start.call_count == 1 and clock.sleeps == 0
assert service["calls"] == []
def test_a_client_bug_is_an_error(self, api_v3_client, service):
with patch(f"{CLIENT}.on_demand_start", side_effect=RuntimeError("boom")):
resp = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
assert resp.status_code == 200
assert resp.status_code == 503
assert resp.get_json()["data"]["socket_error"] == "internal"
assert len(_mailbox_writes(service["cache"])) == 1
assert _mailbox_writes(service["cache"]) == []
def test_an_unknown_reason_is_reported_as_other(self, api_v3_client, service):
# Only known codes are echoed back; anything else stays server-side.
with patch(f"{CLIENT}.on_demand_start",
side_effect=control_client.ControlError("/run/secret/path", "x")):
data = api_v3_client.post(START_URL, json={"plugin_id": "weather"}).get_json()["data"]
assert data["transport"] == "mailbox"
assert data["socket_error"] == "other"
assert len(_mailbox_writes(service["cache"])) == 1
resp = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
assert resp.status_code == 503
assert resp.get_json()["data"]["socket_error"] == "other"
assert "/run/secret" not in resp.get_data(as_text=True)
def test_every_display_error_code_is_reportable(self):
from web_interface.blueprints.api_v3 import display
codes = {v for k, v in vars(c.ErrorCode).items() if not k.startswith("_")}
assert codes <= set(display._REPORTABLE_SOCKET_REASONS)
def test_stop_falls_back(self, api_v3_client, service):
with patch(f"{CLIENT}.on_demand_stop",
side_effect=control_client.ControlError("timeout")):
data = api_v3_client.post(STOP_URL, json={}).get_json()["data"]
assert data["transport"] == "mailbox"
[write] = _mailbox_writes(service["cache"])
assert write == {"request_id": data["request_id"], "action": "stop",
"timestamp": write["timestamp"]}
def test_a_stopped_display_gets_the_mailbox_before_it_is_started(
self, api_v3_client, service):
service["state"]["active"] = False
with patch(f"{CLIENT}.on_demand_start",
side_effect=control_client.ControlError("no_socket")):
resp = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
assert resp.status_code == 200
assert service["calls"] == [("cache", MAILBOX), ("systemctl", "start")]
def test_the_socket_is_off_in_the_test_suite(self, api_v3_client, service):
# conftest's _hermetic_control_socket: a suite run on a device must
# not drive the live display.
assert os.environ[c.SOCKET_PATH_ENV] == "off"
data = api_v3_client.post(START_URL, json={"plugin_id": "weather"}).get_json()["data"]
assert data["transport"] == "mailbox"
assert data["socket_error"] in ("disabled", "unsupported") # Linux, Windows
class TestTheDisplayHadIt:
"""Once the display has the request, its answer stands: no mailbox copy.
"""Once the display has the request, its answer stands.
A busy queue, a refusal or silence after the request was sent mean the
display may have applied it, or would refuse the mailbox copy too, so
the route reports the failure instead of posting it a second time.
display may have applied it, so the route reports the failure and does
not send it again.
"""
@pytest.mark.parametrize("reason,status", [
@@ -217,16 +530,6 @@ class TestTheDisplayHadIt:
stop.assert_called_once()
assert _mailbox_writes(service["cache"]) == []
@pytest.mark.parametrize("reason", ["unknown_command", "unsupported_version"])
def test_an_older_display_that_does_not_speak_it_gets_the_mailbox(
self, api_v3_client, service, reason):
# The upgrade case: new web interface, display still on an old build.
with patch(f"{CLIENT}.on_demand_start",
side_effect=control_client.ControlError(reason, "x", sent=True)):
data = api_v3_client.post(START_URL, json={"plugin_id": "weather"}).get_json()["data"]
assert data["transport"] == "mailbox" and data["socket_error"] == reason
assert len(_mailbox_writes(service["cache"])) == 1
@pytest.mark.skipif(not c.socket_supported(), reason="AF_UNIX sockets are Linux/macOS only")
class TestRealSocket:
@@ -259,11 +562,21 @@ class TestRealSocket:
assert data["transport"] == "socket"
assert [x.request_id for x in live.drain()] == [data["request_id"]]
def test_a_display_that_went_away_falls_back(self, api_v3_client, service, live):
def test_a_display_that_went_away_is_given_up_on(self, api_v3_client, service, live,
monkeypatch):
from web_interface import on_demand_dispatch
monkeypatch.setattr(f"{DISPLAY}.ON_DEMAND_SOCKET_WAIT_RUNNING_SECONDS", 0.3)
monkeypatch.setattr(on_demand_dispatch, "RETRY_INTERVAL", 0.01)
live.close()
data = api_v3_client.post(START_URL, json={"plugin_id": "weather"}).get_json()["data"]
assert data["transport"] == "mailbox" and data["socket_error"] == "no_socket"
assert len(_mailbox_writes(service["cache"])) == 1
resp = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
# The service still reads as running: answered at once, and the
# web process gives up once the wait is over.
assert resp.status_code == 202
assert resp.get_json()["data"]["socket_error"] == "no_socket"
d = on_demand_dispatch.current()
assert _until(lambda: not d.pending())
assert d.status()["error"] == "start-timeout"
assert _mailbox_writes(service["cache"]) == []
def test_a_full_queue_is_reported_not_mailed(self, api_v3_client, service, monkeypatch):
import shutil
test/test_display_pending_changes.py
+36 -37
View File
@@ -6,7 +6,7 @@ a Vegas iteration runs for vegas_scroll.max_cycle_duration (240s here). On a
real Pi on 2026-09-23:
* an on-demand request posted at 10:54:27 was activated at 10:57:24, when the
Vegas iteration it arrived during finally ended -- nothing read the mailbox
Vegas iteration it arrived during finally ended -- nothing read the request
in between, because _check_vegas_interrupt only looked at a flag that the
main-loop read sets;
* two brightness saves 12s apart inside one 30s screen never showed at all.
@@ -15,6 +15,11 @@ _service_pending_changes is the fix: a throttled pass the dwell sleep, the
render loops and the Vegas interrupt check all call. These tests drive those
long stretches on a fake clock and check a change lands within one throttle
interval -- and that the throttle holds, since the callers run at frame rate.
The on-demand requests here come from a plugin in the display process
(submit_plugin_on_demand), the way in that needs no socket; the file mailbox
these tests used to write is gone (stage 5). A queued request skips the
throttle, so it lands at the next pass's call, not the next interval.
"""
import threading
@@ -28,7 +33,6 @@ from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.coordinator import VegasModeCoordinator
VEGAS_ITERATION_SECONDS = 240
REQUEST_KEY = 'display_on_demand_request'
class FakeClock:
@@ -75,26 +79,15 @@ def clock(monkeypatch):
@pytest.fixture
def controller(test_display_controller, clock):
"""A controller at rest: no schedule, full brightness, empty mailbox."""
"""A controller at rest: no schedule, full brightness, nothing queued."""
c = test_display_controller
c._refresh_config_cache({'display': {'hardware': {'brightness': 90}}})
c.current_brightness = 90
c.is_display_active = True
c._check_wifi_status_message = MagicMock(return_value=None)
c.mailbox = {} # what the web process has written
def cache_get(key, *args, **kwargs):
if key == REQUEST_KEY:
return c.mailbox.get('request')
return None
def cache_delete(key):
if key == REQUEST_KEY:
c.mailbox.pop('request', None)
c.cache_manager.get = MagicMock(side_effect=cache_get)
c.cache_manager.delete = MagicMock(side_effect=cache_delete)
c.cache_manager.get = MagicMock(return_value=None)
c.cache_manager.delete = MagicMock()
c.cache_manager.set = MagicMock()
c.display_manager.set_brightness = MagicMock(return_value=True)
c.display_manager.update_display = MagicMock()
@@ -110,11 +103,12 @@ def controller(test_display_controller, clock):
return c
def post_request(controller, request_id='r1', mode='clock'):
controller.mailbox['request'] = {
'request_id': request_id, 'action': 'start',
'plugin_id': mode, 'mode': mode,
}
def post_request(controller, request_id='r1', mode='clock', action='start'):
"""A plugin asking for the screen (or giving it back) from its thread."""
assert controller.submit_plugin_on_demand({
'request_id': request_id, 'action': action,
'plugin_id': mode, 'mode': mode, 'source': 'plugin',
})
def save_brightness(controller, brightness):
@@ -124,9 +118,11 @@ def save_brightness(controller, brightness):
{'display': {'hardware': {'brightness': brightness}}})
def mailbox_reads(controller):
return sum(1 for call in controller.cache_manager.get.call_args_list
if call.args and call.args[0] == REQUEST_KEY)
def count_passes(controller):
"""Count the pending-changes passes that got past the throttle."""
controller._poll_on_demand_requests = MagicMock(
wraps=controller._poll_on_demand_requests)
return controller._poll_on_demand_requests
def vegas_coordinator(controller):
@@ -206,15 +202,16 @@ class TestOnDemandDuringVegas:
"interrupt checker never saw the request")
assert controller.on_demand_active
def test_an_empty_mailbox_lets_the_iteration_run_out(self, controller, clock):
def test_a_quiet_iteration_runs_out(self, controller, clock):
coord = vegas_coordinator(controller)
passes = count_passes(controller)
assert coord.run_iteration() is True
assert not controller.on_demand_active
# And the read is throttled: at most one per interval, not per check.
max_reads = VEGAS_ITERATION_SECONDS / controller.PENDING_CHANGES_INTERVAL + 1
assert 0 < mailbox_reads(controller) <= max_reads
# And the pass is throttled: at most one per interval, not per check.
max_passes = VEGAS_ITERATION_SECONDS / controller.PENDING_CHANGES_INTERVAL + 1
assert 0 < passes.call_count <= max_passes
checks = coord.frames // coord._interrupt_check_interval
assert mailbox_reads(controller) < checks / 2
assert passes.call_count < checks / 2
class TestBrightnessIsAppliedMidScreen:
@@ -294,23 +291,25 @@ class TestBrightnessIsAppliedMidScreen:
class TestThrottle:
def test_no_reads_or_brightness_calls_between_passes(self, controller, clock):
def test_no_passes_or_brightness_calls_between_passes(self, controller, clock):
controller._service_pending_changes()
reads = mailbox_reads(controller)
passes = count_passes(controller)
save_brightness(controller, 40)
post_request(controller)
for _ in range(500): # a few seconds of frames, all inside one interval
controller._service_pending_changes()
clock.t += controller.PENDING_CHANGES_INTERVAL / 1000
assert mailbox_reads(controller) == reads
assert passes.call_count == 0
controller.display_manager.set_brightness.assert_not_called()
assert not controller.on_demand_active
clock.t += controller.PENDING_CHANGES_INTERVAL
controller._service_pending_changes()
# The poll, plus the consume step's re-read of the request it acted on.
assert mailbox_reads(controller) > reads
assert passes.call_count == 1
controller.display_manager.set_brightness.assert_called_once_with(40)
def test_a_queued_request_skips_the_throttle(self, controller, clock):
controller._service_pending_changes()
post_request(controller)
controller._service_pending_changes() # inside the interval
assert controller.on_demand_active
def test_between_passes_it_does_no_work_at_all(self, controller, clock):
@@ -397,7 +396,7 @@ class TestScheduleAndDwells:
controller._service_pending_changes()
assert controller.on_demand_active
clock.t += controller.PENDING_CHANGES_INTERVAL
controller.mailbox['request'] = {'request_id': 'r2', 'action': 'stop'}
post_request(controller, request_id='r2', action='stop')
start = clock.t
controller._sleep_with_plugin_updates(30)
assert not controller.on_demand_active
test/test_error_snapshot_cross_process.py
+138 -211
View File
@@ -24,7 +24,7 @@ sys.path.insert(0, str(Path(__file__).parent.parent))
from src.cache_manager import CacheManager # noqa: E402
from src import error_aggregator as errors # noqa: E402
from src.error_aggregator import ( # noqa: E402
ERROR_CLEAR_REQUEST_KEY, ERROR_SNAPSHOT_KEY, ErrorAggregator,
ERROR_SNAPSHOT_KEY, ErrorAggregator,
ErrorSnapshotPublisher,
)
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
@@ -314,151 +314,36 @@ class TestRoutes:
assert secret not in published, secret
class TestClear:
def test_clear_is_applied_by_the_display_and_republished(self, web, display):
aggregator, publisher, _ = display
_fail(aggregator)
_fail(aggregator)
publisher.tick()
response = web.post("/api/v3/errors/clear", json={"all": True})
assert response.status_code == 200
body = response.get_json()["data"]
assert body["clear_requested"] is True
assert body["cleared_count"] == 2
# The display applies it on its next tick, throttle or not...
assert publisher.tick() is True
assert aggregator.get_error_summary()["total_errors"] == 0
data = _summary(web)
assert data["total_errors"] == 0 and data["clear_pending"] is False
# ...and only once.
assert publisher.tick() is False
def test_summary_hides_cleared_errors_before_the_display_applies_it(self, web, display):
aggregator, publisher, _ = display
for _ in range(5):
_fail(aggregator)
publisher.tick()
web.post("/api/v3/errors/clear", json={"all": True})
# No display tick yet.
data = _summary(web)
assert data["clear_pending"] is True
assert data["total_errors"] == 0
assert data["recent_errors"] == [] and data["active_patterns"] == {}
assert data["plugin_error_counts"] == {}
plugin = web.get("/api/v3/errors/plugin/p1").get_json()["data"]
assert plugin["status"] == "healthy" and plugin["total_errors"] == 0
def test_a_snapshot_written_just_before_the_clear_cannot_bring_errors_back(
self, web, display, shared_cache):
# The race: the display builds a snapshot, the user clicks Clear, and
# the display's write lands after the request.
aggregator, publisher, _ = display
display_cache, _, _ = shared_cache
for _ in range(3):
_fail(aggregator)
stale = aggregator.build_snapshot()
web.post("/api/v3/errors/clear", json={"all": True})
stale["applied_clear_id"] = None
display_cache.set(ERROR_SNAPSHOT_KEY, stale)
assert _summary(web)["total_errors"] == 0
def test_errors_after_the_clear_are_kept(self, web, display):
aggregator, publisher, clock = display
_fail(aggregator, plugin_id="before")
publisher.tick()
web.post("/api/v3/errors/clear", json={"all": True})
# An error lands after the request but before the display applies it.
for record in aggregator._records:
record.timestamp -= timedelta(seconds=5)
_fail(aggregator, plugin_id="after")
publisher.tick()
data = _summary(web)
assert data["plugin_error_counts"] == {"after": {"ValueError": 1}}
assert data["clear_pending"] is False
def test_age_based_clear(self, web, display):
aggregator, publisher, _ = display
_fail(aggregator, plugin_id="old")
_fail(aggregator, plugin_id="old")
for record in aggregator._records:
record.timestamp -= timedelta(hours=3)
_fail(aggregator, plugin_id="new")
publisher.tick()
body = web.post("/api/v3/errors/clear", json={"max_age_hours": 1}).get_json()["data"]
assert body["cleared_count"] == 2
pending = _summary(web)
assert pending["clear_pending"] is True
assert [r["plugin_id"] for r in pending["recent_errors"]] == ["new"]
publisher.tick()
data = _summary(web)
assert data["plugin_error_counts"] == {"new": {"ValueError": 1}}
assert data["total_errors"] == 1
def test_a_narrower_clear_does_not_undo_a_pending_wider_one(self, web, display):
aggregator, publisher, _ = display
for _ in range(3):
_fail(aggregator)
publisher.tick()
web.post("/api/v3/errors/clear", json={"all": True})
web.post("/api/v3/errors/clear", json={"max_age_hours": 24})
assert _summary(web)["total_errors"] == 0
publisher.tick()
assert aggregator.get_error_summary()["total_errors"] == 0
def test_default_body_still_means_older_than_24_hours(self, web, display):
aggregator, publisher, _ = display
_fail(aggregator)
publisher.tick()
response = web.post("/api/v3/errors/clear")
assert response.status_code == 200
assert response.get_json()["data"]["cleared_count"] == 0
publisher.tick()
assert _summary(web)["total_errors"] == 1
def test_validation_is_unchanged_but_all_skips_it(self, web):
assert web.post("/api/v3/errors/clear", json={"max_age_hours": 0}).status_code == 400
assert web.post("/api/v3/errors/clear", json={"max_age_hours": 9000}).status_code == 400
assert web.post("/api/v3/errors/clear",
json={"all": True, "max_age_hours": "junk"}).status_code == 200
def test_a_request_that_did_not_reach_the_cache_is_an_error(self, web, api_v3_module): # noqa: F811
cache = MagicMock()
cache.get.return_value = None
api_v3_module.api_v3.cache_manager = cache
response = web.post("/api/v3/errors/clear", json={"all": True})
assert response.status_code == 500
assert "clear request" in response.get_json()["message"]
CLIENT = "web_interface.blueprints.api_v3.control_client"
RETIRED_CLEAR_KEY = "plugin_error_clear_request"
def _mailbox_file(shared_cache):
_, _, directory = shared_cache
return directory / f"{ERROR_CLEAR_REQUEST_KEY}.json"
return directory / f"{RETIRED_CLEAR_KEY}.json"
class TestClearOverTheSocket:
"""``errors.clear``: the display applies the clear before it answers, and
the mailbox is written only when the socket could not carry it."""
@pytest.fixture
def socket_up(display, monkeypatch):
"""The control socket, as the display serves it: errors_clear runs the
display's own handler against its publisher."""
from src.ipc import client as control_client
from src.ipc.contract import ErrorsClearArgs
_, publisher, _ = display
monkeypatch.setattr(errors, "_snapshot_publisher", publisher)
calls = []
@pytest.fixture
def socket_up(self, display, monkeypatch):
"""The control socket, as the display serves it: errors_clear runs
the display's own handler against its publisher."""
from src.ipc import client as control_client
from src.ipc.contract import ErrorsClearArgs
_, publisher, _ = display
monkeypatch.setattr(errors, "_snapshot_publisher", publisher)
calls = []
def errors_clear(request_id, cutoff, **kw):
calls.append((request_id, cutoff))
return errors.apply_error_clear(request_id, ErrorsClearArgs(cutoff=cutoff))
def errors_clear(request_id, cutoff, **kw):
calls.append((request_id, cutoff))
return errors.apply_error_clear(request_id, ErrorsClearArgs(cutoff=cutoff))
monkeypatch.setattr(f"{CLIENT}.errors_clear", errors_clear)
assert control_client.errors_clear is errors_clear
return calls
monkeypatch.setattr(f"{CLIENT}.errors_clear", errors_clear)
assert control_client.errors_clear is errors_clear
return calls
class TestClear:
"""``errors.clear``: the display applies the clear before it answers."""
def test_socket_clear_is_applied_before_the_answer(self, web, display, socket_up,
shared_cache):
@@ -471,6 +356,7 @@ class TestClearOverTheSocket:
body = response.get_json()
data = body["data"]
assert data["transport"] == "socket" and data["applied"] is True
assert data["clear_requested"] is True
assert data["cleared_count"] == 3
assert body["message"] == "Cleared all errors"
[(request_id, _)] = socket_up
@@ -479,90 +365,134 @@ class TestClearOverTheSocket:
assert aggregator.get_error_summary()["total_errors"] == 0
summary = _summary(web)
assert summary["total_errors"] == 0 and summary["clear_pending"] is False
# And no mailbox file.
assert not _mailbox_file(shared_cache).exists()
# Nothing left for a tick to do.
assert publisher.tick() is False
def test_errors_after_the_clear_are_kept(self, web, display, socket_up):
aggregator, publisher, _ = display
_fail(aggregator, plugin_id="before")
for record in aggregator._records:
record.timestamp -= timedelta(seconds=5)
publisher.tick()
web.post("/api/v3/errors/clear", json={"all": True})
_fail(aggregator, plugin_id="after")
publisher.min_interval = 0
publisher.tick()
data = _summary(web)
assert data["plugin_error_counts"] == {"after": {"ValueError": 1}}
assert data["clear_pending"] is False
def test_age_based_clear(self, web, display, socket_up):
aggregator, publisher, _ = display
_fail(aggregator, plugin_id="old")
_fail(aggregator, plugin_id="old")
for record in aggregator._records:
record.timestamp -= timedelta(hours=3)
_fail(aggregator, plugin_id="new")
publisher.tick()
body = web.post("/api/v3/errors/clear", json={"max_age_hours": 1}).get_json()["data"]
assert body["cleared_count"] == 2
data = _summary(web)
assert data["plugin_error_counts"] == {"new": {"ValueError": 1}}
assert data["total_errors"] == 1
def test_default_body_still_means_older_than_24_hours(self, web, display, socket_up):
aggregator, publisher, _ = display
_fail(aggregator)
publisher.tick()
response = web.post("/api/v3/errors/clear")
assert response.status_code == 200
assert response.get_json()["data"]["cleared_count"] == 0
assert _summary(web)["total_errors"] == 1
def test_validation_is_unchanged_but_all_skips_it(self, web, socket_up):
assert web.post("/api/v3/errors/clear", json={"max_age_hours": 0}).status_code == 400
assert web.post("/api/v3/errors/clear", json={"max_age_hours": 9000}).status_code == 400
assert web.post("/api/v3/errors/clear",
json={"all": True, "max_age_hours": "junk"}).status_code == 200
class TestClearWithoutTheSocket:
"""Stage 5: no mailbox to fall back to. The route says why it failed and
writes nothing; the errors stay as the display last reported them."""
def _post(self, web, display, monkeypatch, error):
monkeypatch.setattr(f"{CLIENT}.errors_clear", MagicMock(side_effect=error))
aggregator, publisher, _ = display
_fail(aggregator)
publisher.tick()
return web.post("/api/v3/errors/clear", json={"all": True})
@pytest.mark.parametrize("reason", ["no_socket", "refused"])
def test_a_stopped_display_is_an_error(self, web, display, shared_cache, monkeypatch,
reason):
from src.ipc import client as control_client
response = self._post(web, display, monkeypatch,
control_client.ControlError(reason, sent=False))
assert response.status_code == 503
body = response.get_json()
assert body["context"]["socket_error"] == reason
assert "not running" in body["message"]
assert not _mailbox_file(shared_cache).exists()
# Nothing hides them: they are still the display's last report.
summary = _summary(web)
assert summary["total_errors"] == 1 and summary["clear_pending"] is False
@pytest.mark.parametrize("reason", ["disabled", "unsupported"])
def test_no_socket_here_is_an_error(self, web, display, shared_cache, monkeypatch, reason):
from src.ipc import client as control_client
response = self._post(web, display, monkeypatch,
control_client.ControlError(reason, sent=False))
assert response.status_code == 503
assert "not available" in response.get_json()["message"]
assert not _mailbox_file(shared_cache).exists()
def test_an_older_mailbox_request_does_not_read_as_pending(self, web, display, socket_up,
shared_cache, monkeypatch):
# A clear that went to the mailbox while the socket was down, then a
# wider one over the socket: the old request has nothing left to hide.
from unittest.mock import patch
def test_an_older_display_is_told_to_restart(self, web, display, shared_cache,
monkeypatch):
from src.ipc import client as control_client
aggregator, publisher, _ = display
_fail(aggregator)
publisher.tick()
with patch(f"{CLIENT}.errors_clear",
side_effect=control_client.ControlError("no_socket")):
data = web.post("/api/v3/errors/clear", json={"max_age_hours": 1}).get_json()["data"]
assert data["transport"] == "mailbox"
assert _mailbox_file(shared_cache).exists()
data = web.post("/api/v3/errors/clear", json={"all": True}).get_json()["data"]
assert data["transport"] == "socket"
assert _summary(web)["clear_pending"] is False
@pytest.mark.parametrize("reason", ["no_socket", "refused", "disabled", "unsupported"])
def test_no_socket_writes_the_mailbox(self, web, display, shared_cache, monkeypatch, reason):
from src.ipc import client as control_client
monkeypatch.setattr(f"{CLIENT}.errors_clear", MagicMock(
side_effect=control_client.ControlError(reason, sent=False)))
aggregator, publisher, _ = display
_fail(aggregator)
publisher.tick()
data = web.post("/api/v3/errors/clear", json={"all": True}).get_json()["data"]
assert data["transport"] == "mailbox" and data["applied"] is False
assert _mailbox_file(shared_cache).exists()
assert _summary(web)["clear_pending"] is True
publisher.tick()
assert aggregator.get_error_summary()["total_errors"] == 0
def test_an_older_display_gets_the_mailbox(self, web, display, shared_cache, monkeypatch):
# The upgrade case: new web interface, a display from before errors.clear.
from src.ipc import client as control_client
monkeypatch.setattr(f"{CLIENT}.errors_clear", MagicMock(
side_effect=control_client.ControlError("unknown_command", sent=True)))
aggregator, publisher, _ = display
_fail(aggregator)
publisher.tick()
data = web.post("/api/v3/errors/clear", json={"all": True}).get_json()["data"]
assert data["transport"] == "mailbox"
publisher.tick()
assert aggregator.get_error_summary()["total_errors"] == 0
response = self._post(web, display, monkeypatch,
control_client.ControlError("unknown_command", sent=True))
assert response.status_code == 503
assert "restart" in response.get_json()["message"]
assert not _mailbox_file(shared_cache).exists()
@pytest.mark.parametrize("reason", ["internal", "timeout", "busy", "invalid_args"])
def test_a_display_that_had_it_and_failed_is_an_error(self, web, display, shared_cache,
monkeypatch, reason):
from src.ipc import client as control_client
monkeypatch.setattr(f"{CLIENT}.errors_clear", MagicMock(
side_effect=control_client.ControlError(reason, sent=True)))
response = web.post("/api/v3/errors/clear", json={"all": True})
response = self._post(web, display, monkeypatch,
control_client.ControlError(reason, sent=True))
assert response.status_code == 503
assert response.get_json()["context"]["socket_error"] == reason
body = response.get_json()
assert body["context"]["socket_error"] == reason
assert body["message"] == "The display service did not apply the clear"
assert not _mailbox_file(shared_cache).exists()
class TestPublisherMailboxPoll:
def test_the_mailbox_is_read_only_when_its_file_changed(self, display, shared_cache):
_, publisher, _ = display
_, web_cache, _ = shared_cache
def test_the_default_test_setup_has_no_socket(self, web, display, shared_cache):
# conftest turns the socket off: the real client answers "disabled".
aggregator, publisher, _ = display
_fail(aggregator)
publisher.tick()
response = web.post("/api/v3/errors/clear", json={"all": True})
assert response.status_code == 503
assert response.get_json()["context"]["socket_error"] in ("disabled", "unsupported")
class TestPublisher:
def test_a_tick_reads_no_clear_request(self, display, shared_cache):
_, publisher, clock = display
_, web_cache, directory = shared_cache
# An old web interface's leftover request file is ignored.
(directory / f"{RETIRED_CLEAR_KEY}.json").write_text(
'{"timestamp": 1, "data": {"request_id": "old", "cutoff": 9e9}}')
publisher.cache_manager = MagicMock(wraps=publisher.cache_manager)
def reads():
return [c for c in publisher.cache_manager.get.call_args_list
if c.args[0] == ERROR_CLEAR_REQUEST_KEY]
for _ in range(5):
clock.now += 60
publisher.tick()
assert reads() == [] # no file: a stat per tick, no read
errors.request_error_clear(web_cache, 1.0)
publisher.tick()
assert len(reads()) == 1
for _ in range(5):
publisher.tick()
assert len(reads()) == 1 # unchanged file: not read again
errors.request_error_clear(web_cache, 2.0)
publisher.tick()
assert len(reads()) == 2
publisher.cache_manager.get.assert_not_called()
snapshot = web_cache.get(ERROR_SNAPSHOT_KEY, max_age=None, memory_ttl=0)
assert snapshot["applied_clear_id"] is None
def test_clear_now_publishes_what_it_applied(self, display, shared_cache):
aggregator, publisher, _ = display
@@ -572,7 +502,6 @@ class TestPublisherMailboxPoll:
snapshot = web_cache.get(ERROR_SNAPSHOT_KEY, max_age=None, memory_ttl=0)
assert snapshot["applied_clear_id"] == "sock-1"
assert snapshot["total_errors"] == 0
assert snapshot["applied_clear_cutoff"] is not None
def test_the_handler_needs_a_running_publisher(self, monkeypatch):
from src.ipc.contract import ErrorsClearArgs
@@ -583,12 +512,10 @@ class TestPublisherMailboxPoll:
@pytest.mark.skipif(not hasattr(os, "fchmod") or os.name == "nt",
reason="POSIX file modes")
def test_both_files_are_group_readable(web, display, shared_cache):
def test_the_snapshot_is_group_readable(web, display, shared_cache):
aggregator, publisher, _ = display
_, _, directory = shared_cache
_fail(aggregator)
publisher.tick()
web.post("/api/v3/errors/clear", json={"all": True})
for key in (ERROR_SNAPSHOT_KEY, ERROR_CLEAR_REQUEST_KEY):
mode = stat.S_IMODE(os.stat(directory / f"{key}.json").st_mode)
assert mode == 0o660, (key, oct(mode))
mode = stat.S_IMODE(os.stat(directory / f"{ERROR_SNAPSHOT_KEY}.json").st_mode)
assert mode == 0o660, oct(mode)
test/test_ipc_contract.py
+3 -3
View File
@@ -3,7 +3,7 @@
Pure data, so every test here runs on every platform. What they pin:
* a request and a response survive encode -> decode -> parse unchanged, and
the on-demand arguments carry exactly what the file mailbox carries;
the on-demand arguments carry exactly what the REST route sends;
* the envelope and the arguments refuse what the display could not act on
(missing ids, wrong types, a non-finite duration) with a stable error code;
* framing never holds more than one message's worth of bytes, however the
@@ -186,8 +186,8 @@ class TestOnDemandArgs:
class TestMailboxShape:
"""Socket commands are handed to the mailbox's own handler, so they must
look exactly like what the web route writes to the mailbox."""
"""Socket commands are handed to the display's on-demand handler, so they
must look exactly like the request dict it takes."""
def test_start(self):
args = OnDemandStartArgs(plugin_id='clock', mode='clock_main', duration=60.0,
test/test_ipc_display_hook.py
+34 -58
View File
@@ -1,14 +1,13 @@
"""DisplayController's side of the control socket.
The server's handlers only queue; the render thread drains the queue where
it reads the file mailbox (_poll_on_demand_requests) and hands each command
to the mailbox's own handler (_handle_on_demand_request). These tests pin
that hook:
The server's handlers only queue; the render thread drains the queue
(_poll_on_demand_requests) and hands each on-demand command to
_handle_on_demand_request, which plugins' own requests use too. These tests
pin that hook:
* a socket command is applied by the same code as a mailbox request, with
its request id, and without waiting for the mailbox's 0.25 s read floor;
* a request that arrives both ways (a client that timed out after the
command was queued, then wrote the mailbox) is activated once;
* a socket command is applied with its request id, at once, and touches no
cache key (the file mailbox and the persisted processed id are gone);
* a start sent twice with one request id is activated once;
* a command that fails is contained, and the ones after it still run;
* cleanup closes the socket; a disabled socket changes nothing.
"""
@@ -58,28 +57,15 @@ def controller(test_display_controller):
c_ = test_display_controller
c_.on_demand_active = False
c_.on_demand_request_id = None
c_._last_on_demand_poll = None
mailbox = {'value': None}
def fake_get(key, *a, **kw):
if key == 'display_on_demand_request':
return mailbox['value']
return None
def fake_delete(key):
if key == 'display_on_demand_request':
mailbox['value'] = None
c_.cache_manager.get = MagicMock(side_effect=fake_get)
c_.cache_manager.get = MagicMock(return_value=None)
c_.cache_manager.set = MagicMock()
c_.cache_manager.delete = MagicMock(side_effect=fake_delete)
c_.cache_manager.delete = MagicMock()
c_._activate_on_demand = MagicMock()
c_.mailbox = mailbox
return c_
class TestDrain:
def test_a_socket_start_goes_through_the_mailbox_handler(self, controller):
def test_a_socket_start_is_activated_with_its_request_id(self, controller):
controller._control_server = FakeServer(_start('sock-1', duration=30.0, pinned=True))
controller._poll_on_demand_requests()
controller._activate_on_demand.assert_called_once()
@@ -89,47 +75,28 @@ class TestDrain:
assert request['plugin_id'] == 'clock'
assert request['duration'] == 30.0 and request['pinned'] is True
assert controller.on_demand_request_id == 'sock-1'
# The same restart-replay guard as a mailbox request.
controller.cache_manager.set.assert_any_call(
'display_on_demand_processed_id', 'sock-1', ttl=3600)
# No mailbox, and no persisted processed id.
keys = {call.args[0] for m in (controller.cache_manager.get,
controller.cache_manager.set,
controller.cache_manager.delete)
for call in m.call_args_list}
assert not keys & {'display_on_demand_request', 'display_on_demand_processed_id'}
def test_socket_commands_skip_the_mailbox_floor(self, controller):
def test_socket_commands_land_at_once(self, controller):
server = FakeServer()
controller._control_server = server
controller._poll_on_demand_requests() # reads the mailbox, sets the floor
reads = controller.cache_manager.get.call_count
controller._poll_on_demand_requests()
server.commands.append(_start('quick'))
controller._poll_on_demand_requests() # within the floor
controller._activate_on_demand.assert_called_once()
mailbox_reads = [call for call in controller.cache_manager.get.call_args_list[reads:]
if call.args[0] == 'display_on_demand_request']
# Only _consume_on_demand_request's compare-before-delete re-read.
assert len(mailbox_reads) <= 1
def test_a_request_that_came_both_ways_is_activated_once(self, controller):
controller._control_server = FakeServer(_start('dup'))
controller.mailbox['value'] = {'request_id': 'dup', 'action': 'start',
'plugin_id': 'clock'}
controller._poll_on_demand_requests()
controller._last_on_demand_poll = None
controller._poll_on_demand_requests()
controller._activate_on_demand.assert_called_once()
assert controller.mailbox['value'] is None, "the duplicate was left in the mailbox"
def test_a_fallback_write_landing_later_is_ignored(self, controller):
controller._control_server = FakeServer(_start('late'))
controller._poll_on_demand_requests()
controller.mailbox['value'] = {'request_id': 'late', 'action': 'start',
'plugin_id': 'clock'}
controller._last_on_demand_poll = None
controller._poll_on_demand_requests()
controller._activate_on_demand.assert_called_once()
def test_the_mailbox_still_works_alongside(self, controller):
controller._control_server = FakeServer()
controller.mailbox['value'] = {'request_id': 'mb', 'action': 'start', 'plugin_id': 'p'}
def test_a_start_sent_twice_is_activated_once(self, controller):
server = FakeServer(_start('dup'))
controller._control_server = server
controller._poll_on_demand_requests()
assert controller._activate_on_demand.call_args.args[0]['request_id'] == 'mb'
server.commands.append(_start('dup'))
controller._poll_on_demand_requests()
controller._activate_on_demand.assert_called_once()
def test_a_socket_stop_ends_on_demand(self, controller):
controller.on_demand_active = True
@@ -159,10 +126,11 @@ class TestDrain:
controller._poll_on_demand_requests()
assert calls == ['bad', 'good']
def test_no_server_means_mailbox_only(self, controller):
def test_no_server_means_nothing_to_apply(self, controller):
controller._control_server = None
controller._poll_on_demand_requests()
controller._activate_on_demand.assert_not_called()
controller.cache_manager.get.assert_not_called()
class TestPendingChangesFloor:
@@ -194,6 +162,14 @@ class TestLifecycle:
assert status['on_demand']['plugin_id'] == 'clock'
c.encode_message(status) # it has to fit on the wire
def test_the_state_names_the_request_it_answers(self, controller):
# The web interface keeps reporting a delivered start as "starting"
# until the display publishes state for that request id.
assert controller._on_demand_state()['request_id'] is None
controller._control_server = FakeServer(_start('sock-7'))
controller._poll_on_demand_requests()
assert controller._on_demand_state()['request_id'] == 'sock-7'
def test_cleanup_closes_the_socket(self, controller):
server = FakeServer()
controller._control_server = server
test/test_ipc_stage4.py
+141 -206
View File
@@ -1,14 +1,13 @@
"""Stage 4 of the control socket: the file mailboxes are only a fallback.
"""Stages 4 and 5 of the control socket: the socket is the only way in.
* The client knows whether the display had the request (``ControlError.sent``)
and ``should_fall_back`` allows a mailbox write only when it did not, or
when the display is too old to know the command (the upgrade case).
and ``display_not_listening`` says when no display was there to take it
(stopped or still starting), the one case a later retry can fix.
* ``errors.clear`` is answered on the connection thread by a handler the
display registers; a display without one answers like an older display.
* The display looks at the on-demand mailbox once a second while the socket
is up (0.25 s without it), reads it only when its file changed, never
touches it for a socket command, and logs who still writes it.
* ``CacheManager.file_signature`` / ``MailboxWatch`` make a look one stat().
* Stage 5: the display no longer reads the ``display_on_demand_request``
mailbox at all, and the cache refuses writes to the retired mailbox keys,
warning once per writer.
The web routes are covered in test_api_v3_on_demand_socket.py and
test_error_snapshot_cross_process.py.
@@ -23,7 +22,8 @@ from unittest.mock import MagicMock, patch
import pytest
from src.cache_manager import CacheManager, MailboxWatch
from src import cache_manager as cache_module
from src.cache_manager import CacheManager
from src.ipc import client
from src.ipc import contract as c
from src.ipc.contract import Command, ErrorsClearArgs, OnDemandStartArgs, ProtocolError
@@ -81,38 +81,41 @@ class TestSent:
def test_an_answer_is_returned(self):
assert _call(FakeSock([_reply('rid-1', ok=True, result={'pong': True})])) == {'pong': True}
@pytest.mark.parametrize('reason', ['no_socket', 'refused', 'timeout', 'busy'])
def test_a_failed_connect_was_not_sent(self, reason):
@pytest.mark.parametrize('reason,listening', [('no_socket', False), ('refused', False),
('timeout', True), ('busy', True)])
def test_a_failed_connect_was_not_sent(self, reason, listening):
e = _error(connect_error=client.ControlError(reason))
assert e.reason == reason and e.sent is False
assert client.should_fall_back(e)
# Only "nothing there" is worth waiting for: a timeout or a full
# backlog is a display that is there and stuck.
assert client.display_not_listening(e) is not listening
def test_a_send_that_timed_out_was_not_sent(self):
e = _error(sock=FakeSock(send_error=socket.timeout()))
assert e.reason == 'timeout' and e.sent is False
assert client.should_fall_back(e)
assert not client.display_not_listening(e)
def test_silence_after_the_request_was_sent(self):
e = _error(sock=FakeSock(recv_error=socket.timeout()))
assert e.reason == 'timeout' and e.sent is True
assert not client.should_fall_back(e)
assert not client.display_not_listening(e)
def test_a_hang_up_after_the_request_was_sent(self):
e = _error(sock=FakeSock([]))
assert e.reason == 'closed' and e.sent is True
assert not client.should_fall_back(e)
assert not client.display_not_listening(e)
def test_a_garbled_reply(self):
e = _error(sock=FakeSock([b'not json\n']))
assert e.reason == 'bad_response' and e.sent is True
assert not client.should_fall_back(e)
assert not client.display_not_listening(e)
@pytest.mark.parametrize('code', ['busy', 'invalid_args', 'internal', 'pending', 'failed'])
def test_a_display_error_with_an_id_was_sent(self, code):
sock = FakeSock([_reply('rid-1', ok=False, error={'code': code, 'message': 'x'})])
e = _error(sock=sock)
assert e.reason == code and e.sent is True
assert not client.should_fall_back(e)
assert not client.display_not_listening(e)
@pytest.mark.parametrize('code', ['forbidden', 'busy'])
def test_a_refusal_at_the_door_was_not_sent(self, code):
@@ -122,23 +125,33 @@ class TestSent:
'error': {'code': code, 'message': 'x'}}) + '\n').encode()])
e = _error(sock=sock)
assert e.reason == code and e.sent is False
assert client.should_fall_back(e)
assert not client.display_not_listening(e)
@pytest.mark.parametrize('code', ['unknown_command', 'unsupported_version'])
def test_an_older_display_is_fallen_back_from(self, code):
def test_an_older_display_is_listening(self, code):
sock = FakeSock([_reply('rid-1', ok=False, error={'code': code, 'message': 'x'})])
e = _error(sock=sock)
assert e.sent is True
assert client.should_fall_back(e)
assert not client.display_not_listening(e)
def test_a_request_refused_locally_never_left(self):
with pytest.raises(client.ControlError) as e:
client.request(Command.ERRORS_CLEAR, {'cutoff': 'soon'}, paths=['/x.sock'])
assert e.value.reason == 'invalid_request' and e.value.sent is False
assert client.should_fall_back(e.value)
assert not client.display_not_listening(e.value)
def test_a_client_bug_falls_back(self):
assert client.should_fall_back(RuntimeError('boom'))
@pytest.mark.parametrize('reason', ['disabled', 'unsupported'])
def test_a_client_without_the_socket_is_not_waiting_for_a_display(self, reason):
assert not client.display_not_listening(client.ControlError(reason))
@pytest.mark.parametrize('reason', sorted(client.NOT_LISTENING_REASONS))
def test_a_request_that_was_sent_had_a_display(self, reason):
# Only a request that never left can be waited on and sent again: one
# that was sent may have been applied.
assert not client.display_not_listening(client.ControlError(reason, sent=True))
def test_a_client_bug_is_not_a_missing_display(self):
assert not client.display_not_listening(RuntimeError('boom'))
# -- errors.clear on the server ------------------------------------------------------
@@ -204,37 +217,28 @@ class TestErrorsClearOnTheServer:
assert Command.ERRORS_CLEAR in result['commands']
# -- the display's mailbox poll ------------------------------------------------------
# -- stage 5: the display reads no mailbox --------------------------------------------
class SignedCache:
"""The slice of CacheManager the poll uses, counting what it costs."""
class CountingCache:
"""The slice of CacheManager the on-demand path uses, counting every call."""
def __init__(self):
self.data = {}
self.writes = 0
self.version = {}
self.reads = []
self.stats = 0
self.deletes = []
self.sets = []
self.calls = []
def file_signature(self, key):
self.stats += 1
return (self.version[key], 0, 0) if key in self.data else None
def __getattr__(self, name):
# Any other method (file_signature, delete, ...) is recorded too.
def call(*a, **kw):
self.calls.append((name, a[0] if a else None))
return call
def get(self, key, *a, **kw):
self.reads.append(key)
self.calls.append(('get', key))
return self.data.get(key)
def set(self, key, value, *a, **kw):
self.sets.append(key)
self.calls.append(('set', key))
self.data[key] = value
self.writes += 1
self.version[key] = self.writes
def delete(self, key):
self.deletes.append(key)
self.data.pop(key, None)
class FakeServer:
@@ -250,209 +254,140 @@ class FakeServer:
return out
class Clock:
def __init__(self):
self.t = 1000.0
def __call__(self):
return self.t
@pytest.fixture
def controller(test_display_controller, monkeypatch):
def controller(test_display_controller):
dc = test_display_controller
dc.cache_manager = SignedCache()
dc.cache_manager = CountingCache()
dc._activate_on_demand = MagicMock()
dc.on_demand_active = False
dc.on_demand_request_id = None
dc._last_on_demand_poll = None
dc._on_demand_mailbox = None
dc._mailbox_writers_logged = frozenset()
clock = Clock()
monkeypatch.setattr('src.display_controller.time.monotonic', clock)
dc.clock = clock
return dc
def _post(dc, rid, action='start', **fields):
dc.cache_manager.set(MAILBOX, dict({'request_id': rid, 'action': action}, **fields))
def _poll_for(dc, seconds, step=1 / 16): # exact in binary: no drift past a floor
end = dc.clock.t + seconds
while dc.clock.t < end:
dc._poll_on_demand_requests()
dc.clock.t += step
class TestMailboxCadence:
def test_without_a_socket_it_is_looked_at_every_quarter_second(self, controller):
controller._control_server = None
_poll_for(controller, 10.0)
assert 38 <= controller.cache_manager.stats <= 42
def test_with_a_socket_it_is_looked_at_once_a_second(self, controller):
controller._control_server = FakeServer()
_poll_for(controller, 10.0)
assert 9 <= controller.cache_manager.stats <= 11
def test_a_look_that_finds_nothing_reads_nothing(self, controller):
controller._control_server = FakeServer()
_poll_for(controller, 10.0)
assert controller.cache_manager.reads == []
def test_an_unchanged_mailbox_is_not_read_again(self, controller):
# An already-processed start the delete could not remove, say.
controller._control_server = FakeServer()
controller.cache_manager.delete = MagicMock() # the file stays
_post(controller, 'once', plugin_id='clock')
_poll_for(controller, 10.0)
assert controller.cache_manager.reads.count(MAILBOX) <= 2 # the read + the re-check
controller._activate_on_demand.assert_called_once()
def test_a_mailbox_request_lands_within_a_second_with_the_socket_up(self, controller):
# The upgrade case the other way round: a new display, and a web
# interface (or a plugin) that still writes the mailbox.
controller._control_server = FakeServer()
controller._poll_on_demand_requests()
controller.clock.t += 0.1
_post(controller, 'old-web', plugin_id='clock')
posted = controller.clock.t
while not controller._activate_on_demand.called:
class TestNoMailbox:
@pytest.mark.parametrize('with_socket', [False, True])
def test_polling_touches_no_cache_key(self, controller, with_socket):
controller._control_server = FakeServer() if with_socket else None
controller.cache_manager.data[MAILBOX] = {'request_id': 'left', 'action': 'start',
'plugin_id': 'clock'}
for _ in range(200):
controller._poll_on_demand_requests()
controller.clock.t += 0.05
assert controller.clock.t - posted < 1.5
assert controller.clock.t - posted <= controller.MAILBOX_POLL_INTERVAL_WITH_SOCKET + 0.06
assert MAILBOX in controller.cache_manager.deletes # consumed
assert controller.cache_manager.calls == []
controller._activate_on_demand.assert_not_called()
def test_socket_commands_still_land_at_once(self, controller):
def test_socket_commands_land_at_once(self, controller):
server = controller._control_server = FakeServer()
controller._poll_on_demand_requests()
server.commands.append(QueuedCommand('sock', Command.ON_DEMAND_START,
OnDemandStartArgs(plugin_id='clock'), time.time()))
controller._poll_on_demand_requests() # inside the mailbox interval
controller._poll_on_demand_requests()
controller._activate_on_demand.assert_called_once()
class TestSocketCommandsLeaveTheMailboxAlone:
def test_a_socket_start_reads_and_deletes_no_mailbox(self, controller):
def test_a_start_is_applied_once_and_writes_no_processed_id(self, controller):
server = controller._control_server = FakeServer()
controller._poll_on_demand_requests()
before = list(controller.cache_manager.reads)
server.commands.append(QueuedCommand('s1', Command.ON_DEMAND_START,
OnDemandStartArgs(plugin_id='clock'), time.time()))
controller._poll_on_demand_requests()
for _ in range(2):
server.commands.append(QueuedCommand('same', Command.ON_DEMAND_START,
OnDemandStartArgs(plugin_id='clock'), time.time()))
controller._poll_on_demand_requests()
controller._activate_on_demand.assert_called_once()
assert MAILBOX not in controller.cache_manager.reads[len(before):]
assert controller.cache_manager.deletes == []
assert controller.cache_manager.calls == []
def test_a_socket_stop_reads_and_deletes_no_mailbox(self, controller):
def test_a_socket_stop_touches_no_cache_key(self, controller):
from src.ipc.contract import OnDemandStopArgs
controller.on_demand_active = True
controller._clear_on_demand = MagicMock()
server = controller._control_server = FakeServer()
server.commands.append(QueuedCommand('s2', Command.ON_DEMAND_STOP,
OnDemandStopArgs(), time.time()))
controller.clock.t += 5
controller._drain_control_commands()
controller._clear_on_demand.assert_called_once()
assert MAILBOX not in controller.cache_manager.reads
assert controller.cache_manager.deletes == []
assert controller.cache_manager.calls == []
def test_a_mailbox_copy_of_a_socket_command_is_dropped(self, controller):
# An older web interface timed out after the display queued the
# command, then wrote the mailbox too.
server = controller._control_server = FakeServer()
server.commands.append(QueuedCommand('both', Command.ON_DEMAND_START,
OnDemandStartArgs(plugin_id='clock'), time.time()))
controller._poll_on_demand_requests()
_post(controller, 'both', plugin_id='clock')
_poll_for(controller, 2.0)
controller._activate_on_demand.assert_called_once()
assert MAILBOX not in controller.cache_manager.data
def test_the_mailbox_helpers_are_gone(self):
from src import display_controller as dcm
from src import error_aggregator as ea
for name in ('ON_DEMAND_MAILBOX_KEY', 'MailboxWatch'):
assert not hasattr(dcm, name)
assert not hasattr(ea, 'ERROR_CLEAR_REQUEST_KEY')
assert not hasattr(cache_module, 'MailboxWatch')
assert not hasattr(CacheManager, 'file_signature')
assert not hasattr(client, 'should_fall_back')
for name in ('_consume_on_demand_request', '_note_mailbox_request',
'_mailbox_poll_interval', 'MAILBOX_POLL_INTERVAL_WITH_SOCKET'):
assert not hasattr(dcm.DisplayController, name)
class TestDeprecationLog:
def test_each_mailbox_writer_is_logged_once(self, controller, caplog):
controller._control_server = FakeServer()
caplog.set_level(logging.INFO, logger='src.display_controller')
for i, plugin in enumerate(['on-air', 'on-air', 'pomodoro-timer']):
_post(controller, f'r{i}', plugin_id=plugin)
_poll_for(controller, 1.2)
lines = [r.getMessage() for r in caplog.records if 'file mailbox' in r.getMessage()]
assert len(lines) == 2
assert 'on-air' in lines[0] and 'pomodoro-timer' in lines[1]
def test_nothing_is_logged_without_a_socket(self, controller, caplog):
controller._control_server = None
caplog.set_level(logging.INFO, logger='src.display_controller')
_post(controller, 'r', plugin_id='on-air')
_poll_for(controller, 1.0)
controller._activate_on_demand.assert_called_once()
assert not [r for r in caplog.records if 'file mailbox' in r.getMessage()]
# -- file_signature and MailboxWatch -------------------------------------------------
# -- stage 5: writes to the retired keys are refused, with one warning per writer -----
@pytest.fixture
def real_cache(tmp_path, monkeypatch):
monkeypatch.setattr(CacheManager, '_get_writable_cache_dir', lambda self: str(tmp_path))
monkeypatch.setattr(cache_module, '_retired_writers_warned', set())
cache = CacheManager()
yield cache
cache.stop_cleanup_thread()
class TestFileSignature:
def test_absent_key(self, real_cache):
assert real_cache.file_signature('nothing') is None
class OldPlugin:
"""What an old plugin looks like on the stack: BasePlugin gives every
plugin ``plugin_id`` and ``cache_manager``."""
def test_every_write_is_a_new_signature(self, real_cache):
seen = set()
for i in range(20):
# Same size each time, written as fast as possible.
real_cache.set(MAILBOX, {'request_id': f'r{i:02d}'})
sig = real_cache.file_signature(MAILBOX)
assert isinstance(sig, tuple)
seen.add(sig)
assert len(seen) == 20
def __init__(self, plugin_id, cache):
self.plugin_id = plugin_id
self.cache_manager = cache
def test_gone_after_a_delete(self, real_cache):
real_cache.set(MAILBOX, {'a': 1})
real_cache.delete(MAILBOX)
assert real_cache.file_signature(MAILBOX) is None
def trigger(self, target=None):
self.cache_manager.set(MAILBOX, {'request_id': 'r', 'action': 'start',
'plugin_id': target or self.plugin_id})
class TestMailboxWatch:
def test_reads_once_per_write(self, real_cache):
watch = MailboxWatch(MAILBOX)
assert watch.changed(real_cache) is False # no file
real_cache.set(MAILBOX, {'request_id': 'a'})
assert watch.changed(real_cache) is True
assert watch.changed(real_cache) is False
real_cache.set(MAILBOX, {'request_id': 'b'})
assert watch.changed(real_cache) is True
def _warnings(caplog):
return [r.getMessage() for r in caplog.records
if r.levelno == logging.WARNING and 'retired' in r.getMessage()]
def test_forget_reads_again(self, real_cache):
watch = MailboxWatch(MAILBOX)
real_cache.set(MAILBOX, {'request_id': 'a'})
assert watch.changed(real_cache) is True
watch.forget()
assert watch.changed(real_cache) is True
def test_a_rewrite_after_a_delete_is_seen(self, real_cache):
watch = MailboxWatch(MAILBOX)
real_cache.set(MAILBOX, {'request_id': 'a'})
assert watch.changed(real_cache)
real_cache.delete(MAILBOX)
assert watch.changed(real_cache) is False
real_cache.set(MAILBOX, {'request_id': 'a'})
assert watch.changed(real_cache) is True
class TestRetiredKeys:
@pytest.mark.parametrize('key', sorted(cache_module.RETIRED_MAILBOX_KEYS))
def test_a_write_stores_nothing(self, real_cache, key):
real_cache.set(key, {'request_id': 'r'})
real_cache.save_cache(key, {'request_id': 'r2'})
assert real_cache.get(key, max_age=None, memory_ttl=0) is None
assert not [f for f in os.listdir(real_cache.cache_dir) if key in f]
def test_a_cache_that_cannot_tell_is_read_every_time(self):
watch = MailboxWatch(MAILBOX)
assert watch.changed(MagicMock()) is True
assert watch.changed(MagicMock()) is True
assert watch.changed(object()) is True
def test_other_keys_are_unaffected(self, real_cache):
real_cache.set('display_on_demand_state', {'active': True})
assert real_cache.get('display_on_demand_state', max_age=None,
memory_ttl=0) == {'active': True}
def test_the_writing_plugin_is_named_once(self, real_cache, caplog):
caplog.set_level(logging.WARNING)
on_air = OldPlugin('on-air', real_cache)
for _ in range(3):
on_air.trigger()
OldPlugin('pomodoro-timer', real_cache).trigger()
lines = _warnings(caplog)
assert len(lines) == 2
assert "plugin 'on-air'" in lines[0] and 'display_on_demand_request' in lines[0]
assert 'request_on_demand' in lines[0]
assert "plugin 'pomodoro-timer'" in lines[1]
def test_the_writer_is_the_caller_not_the_target(self, real_cache, caplog):
caplog.set_level(logging.WARNING)
OldPlugin('mqtt-notifications', real_cache).trigger(target='clock')
(line,) = _warnings(caplog)
assert "plugin 'mqtt-notifications'" in line and 'clock' not in line
def test_without_a_plugin_on_the_stack_the_request_names_it(self, real_cache, caplog):
caplog.set_level(logging.WARNING)
real_cache.set(MAILBOX, {'request_id': 'r', 'action': 'start', 'plugin_id': 'gif-player'})
(line,) = _warnings(caplog)
assert "plugin 'gif-player' (named in the request)" in line
def test_otherwise_unknown(self, real_cache, caplog):
caplog.set_level(logging.WARNING)
real_cache.set('plugin_error_clear_request', {'request_id': 'r', 'cutoff': 1.0})
real_cache.set('plugin_error_clear_request', {'request_id': 'r2', 'cutoff': 2.0})
(line,) = _warnings(caplog)
assert 'by unknown' in line and '/api/v3/errors/clear' in line
# -- end to end over a real socket ---------------------------------------------------
@@ -479,24 +414,24 @@ class TestOverTheSocket:
finally:
server.close()
def test_an_older_display_is_an_upgrade_fallback(self, sock_path):
def test_an_older_display_is_listening(self, sock_path):
server = ControlServer(sock_path) # no errors.clear handler
assert server.start()
try:
with pytest.raises(client.ControlError) as e:
client.errors_clear('clr-2', 1.0, paths=[sock_path])
assert e.value.reason == 'unknown_command' and e.value.sent is True
assert client.should_fall_back(e.value)
assert not client.display_not_listening(e.value)
finally:
server.close()
def test_no_display_is_a_fallback(self, sock_path):
def test_no_display_is_not_listening(self, sock_path):
with pytest.raises(client.ControlError) as e:
client.errors_clear('clr-3', 1.0, paths=[sock_path])
assert e.value.reason == 'no_socket' and e.value.sent is False
assert client.should_fall_back(e.value)
assert client.display_not_listening(e.value)
def test_a_full_queue_is_not_a_fallback(self, sock_path):
def test_a_full_queue_is_listening(self, sock_path):
server = ControlServer(sock_path, queue_size=1)
assert server.start()
try:
@@ -504,6 +439,6 @@ class TestOverTheSocket:
with pytest.raises(client.ControlError) as e:
client.on_demand_start('q2', 'clock', None, paths=[sock_path])
assert e.value.reason == 'busy' and e.value.sent is True
assert not client.should_fall_back(e.value)
assert not client.display_not_listening(e.value)
finally:
server.close()
test/test_mqtt_bridge.py
+24
View File
@@ -327,3 +327,27 @@ class TestCleartextIsCalledOut:
def test_tls_on_does_not_warn(self, bridge_module):
assert bridge_module.warn_if_cleartext(
{"mqtt_tls": True, "mqtt_password": "hunter2"}) is False
class TestTheApiClientReadsStartingAsTaken:
"""A cold start answers 202 with ``status: "starting"``: the request is
taken and the web process delivers it once the display listens. The
bridge must report that as success, not as a failure."""
def _client(self, bridge_module, status_code, body):
response = MagicMock(status_code=status_code)
response.json.return_value = body
session = MagicMock()
session.request.return_value = response
return bridge_module.LEDMatrixClient("http://pi:5000", session=session)
def test_202_starting_is_returned_not_raised(self, bridge_module):
api = self._client(bridge_module, 202, {
"status": "starting", "message": "starting",
"data": {"request_id": "r1", "pending": True}})
assert api.start_on_demand(mode="clock") == {"request_id": "r1", "pending": True}
def test_an_error_is_still_raised(self, bridge_module):
api = self._client(bridge_module, 503, {"status": "error", "message": "no display"})
with pytest.raises(RuntimeError, match="no display"):
api.start_on_demand(mode="clock")
test/test_on_demand_disabled_plugin.py
+2 -7
View File
@@ -346,13 +346,8 @@ class TestResumingAfterTheSession:
class TestStopClearsAnError:
def _post_stop(self, c):
stop = {'request_id': 'S1', 'action': 'stop'}
c._last_on_demand_poll = None
c.cache_manager.get = MagicMock(
side_effect=lambda key, *a, **kw:
stop if key == 'display_on_demand_request' else None)
c.cache_manager.delete = MagicMock()
c._poll_on_demand_requests()
c._handle_on_demand_request({'request_id': 'S1', 'action': 'stop',
'source': 'socket'})
def test_a_stop_after_a_failed_request_clears_the_error(self, controller):
_start(controller, plugin_id='uninstalled')
test/test_on_demand_dispatch.py
+272
View File
@@ -0,0 +1,272 @@
"""The web process's on-demand dispatcher (web_interface/on_demand_dispatch.py).
A start that finds no display listening -- the service was just started, or
is still loading its plugins -- is answered at once (202), and the
dispatcher's one worker thread sends it again until the display
acknowledges it or the wait runs out. These tests drive the worker with a
fake ``send`` and short waits:
* acknowledged: delivered once, and reported as such;
* nothing listening for the whole wait: ``start-timeout``;
* any other failure: reported at once, not retried;
* a newer start supersedes the pending one; a stop cancels it;
* the outcome is reported for a while, then forgotten.
"""
import threading
import time
import pytest
from src.ipc import client as control_client
from web_interface import on_demand_dispatch
from web_interface.on_demand_dispatch import OnDemandDispatcher
def _not_listening():
return control_client.ControlError("no_socket", "x", sent=False)
def _payload(rid, plugin_id="weather"):
return {"request_id": rid, "action": "start", "plugin_id": plugin_id,
"mode": plugin_id, "duration": 30, "pinned": False}
class FakeSend:
"""Answers with ``outcomes`` in turn (an exception is raised, anything
else acks); the last one repeats. Records the request ids it was sent."""
def __init__(self, *outcomes):
self.outcomes = list(outcomes) or ["ack"]
self.sent = []
self.lock = threading.Lock()
def __call__(self, payload):
with self.lock:
self.sent.append(payload["request_id"])
outcome = self.outcomes.pop(0) if len(self.outcomes) > 1 else self.outcomes[0]
if isinstance(outcome, BaseException):
raise outcome
if callable(outcome):
return outcome(payload)
return {"accepted": True}
def _until(predicate, timeout=5.0):
end = time.monotonic() + timeout
while time.monotonic() < end:
if predicate():
return True
time.sleep(0.005)
return False
def _settled(d):
return _until(lambda: not d.pending() and d._thread is None)
@pytest.fixture
def make():
made = []
def build(send, wait_seconds=2.0, retry_interval=0.01):
d = OnDemandDispatcher(send, wait_seconds=wait_seconds, retry_interval=retry_interval)
made.append(d)
return d
yield build
for d in made:
d.cancel("teardown")
class TestDelivery:
def test_an_acknowledged_start_is_delivered_once(self, make):
send = FakeSend("ack")
d = make(send)
d.submit(_payload("r1"))
assert _settled(d)
assert send.sent == ["r1"]
status = d.status()
assert status["status"] == "delivered" and status["request_id"] == "r1"
assert status["source"] == "web"
def test_it_is_sent_again_until_the_display_listens(self, make):
send = FakeSend(_not_listening(), _not_listening(), "ack")
d = make(send)
d.submit(_payload("r1"))
assert _settled(d)
assert send.sent == ["r1", "r1", "r1"]
assert d.status()["status"] == "delivered"
def test_while_it_waits_it_reports_starting(self, make):
gate = threading.Event()
def blocked(payload):
gate.wait(5)
raise _not_listening()
send = FakeSend(blocked, "ack")
d = make(send)
d.submit(_payload("r1"))
status = d.status()
assert status["status"] == "starting" and status["active"] is False
assert (status["plugin_id"], status["mode"], status["duration"]) == ("weather", "weather", 30)
gate.set()
assert _settled(d)
assert d.status()["status"] == "delivered"
class TestGivingUp:
def test_nothing_listening_for_the_whole_wait_is_a_start_timeout(self, make):
send = FakeSend(_not_listening())
d = make(send, wait_seconds=0.2)
started = time.monotonic()
d.submit(_payload("r1"))
assert _settled(d)
waited = time.monotonic() - started
status = d.status()
assert status["status"] == "error" and status["error"] == "start-timeout"
assert 0.15 <= waited < 2.0
assert len(send.sent) > 3 # it kept trying in between
def test_a_start_can_carry_its_own_wait(self, make):
# The route passes a shorter wait for a service that was already
# running; the dispatcher's default must not override it.
d = make(FakeSend(_not_listening()), wait_seconds=30.0)
d.submit(_payload("r1"), wait_seconds=0.1)
assert _until(lambda: not d.pending(), timeout=3.0), "it waited the default"
assert d.status()["error"] == "start-timeout"
@pytest.mark.parametrize("reason,sent", [("busy", True), ("unknown_command", True),
("timeout", True), ("forbidden", False)])
def test_any_other_failure_is_reported_at_once(self, make, reason, sent):
send = FakeSend(control_client.ControlError(reason, "x", sent=sent))
d = make(send)
d.submit(_payload("r1"))
assert _settled(d)
assert send.sent == ["r1"]
status = d.status()
assert status["status"] == "error" and status["error"] == reason
def test_a_client_bug_is_internal(self, make):
d = make(FakeSend(RuntimeError("boom")))
d.submit(_payload("r1"))
assert _settled(d)
assert d.status()["error"] == "internal"
class TestOneAtATime:
def test_a_newer_start_supersedes_the_pending_one(self, make):
send = FakeSend(_not_listening())
d = make(send)
d.submit(_payload("old"))
assert _until(lambda: "old" in send.sent)
d.submit(_payload("new", plugin_id="clock"))
assert d.status()["request_id"] == "new"
n = len(send.sent)
send.outcomes = ["ack"]
assert _settled(d)
assert send.sent[n:] and set(send.sent[n + 1:]) <= {"new"}
assert send.sent[-1] == "new"
status = d.status()
assert status["status"] == "delivered" and status["plugin_id"] == "clock"
def test_an_answer_for_a_superseded_start_is_not_reported(self, make):
# The old start's send is in flight when the new one arrives; its
# ack must not mark the new one delivered.
in_flight, release = threading.Event(), threading.Event()
def slow(payload):
in_flight.set()
release.wait(5)
return {"accepted": True}
send = FakeSend(slow, _not_listening())
d = make(send, wait_seconds=0.3)
d.submit(_payload("old"))
assert in_flight.wait(5)
d.submit(_payload("new"))
release.set()
assert _settled(d)
status = d.status()
assert status["request_id"] == "new"
assert status["status"] == "error" and status["error"] == "start-timeout"
def test_a_stop_cancels_the_pending_start(self, make):
send = FakeSend(_not_listening())
d = make(send)
d.submit(_payload("r1"))
assert _until(lambda: send.sent)
assert d.cancel("requested-stop") == "r1"
assert _settled(d)
n = len(send.sent)
time.sleep(0.05)
assert len(send.sent) == n, "it kept sending a cancelled start"
status = d.status()
assert status["status"] == "idle" and status["last_event"] == "requested-stop"
def test_a_cancel_waits_for_a_send_in_flight(self, make):
# Whatever the caller sends after cancel() must land after the
# cancelled start, not race it to the display.
in_flight, release = threading.Event(), threading.Event()
def slow(payload):
in_flight.set()
release.wait(5)
raise _not_listening()
send = FakeSend(slow)
d = make(send)
d.submit(_payload("old"))
assert in_flight.wait(5)
done = threading.Event()
threading.Thread(target=lambda: (d.cancel("superseded"), done.set()),
daemon=True).start()
assert not done.wait(0.1), "cancel returned while the old send was in flight"
release.set()
assert done.wait(5)
assert _settled(d)
assert send.sent == ["old"]
def test_a_cancel_with_nothing_pending_does_nothing(self, make):
d = make(FakeSend("ack"))
assert d.cancel() is None
assert d.status() is None
def test_a_cancel_after_delivery_leaves_the_outcome(self, make):
d = make(FakeSend("ack"))
d.submit(_payload("r1"))
assert _settled(d)
assert d.cancel() is None
assert d.status()["status"] == "delivered"
class TestOutcomeLifetime:
def test_an_outcome_is_forgotten_after_a_while(self, make, monkeypatch):
d = make(FakeSend("ack"))
d.submit(_payload("r1"))
assert _settled(d)
assert d.status() is not None
monkeypatch.setattr(on_demand_dispatch, "OUTCOME_SECONDS", 0.0)
time.sleep(0.01)
assert d.status() is None
def test_a_pending_start_never_expires(self, make, monkeypatch):
monkeypatch.setattr(on_demand_dispatch, "OUTCOME_SECONDS", 0.0)
gate = threading.Event()
d = make(FakeSend(lambda p: gate.wait(5) and {"accepted": True}))
d.submit(_payload("r1"))
time.sleep(0.01)
assert d.status()["status"] == "starting"
gate.set()
assert _settled(d)
def test_the_process_has_one_dispatcher():
on_demand_dispatch.reset_for_tests()
try:
assert on_demand_dispatch.current() is None
first = on_demand_dispatch.get_dispatcher(FakeSend())
assert on_demand_dispatch.get_dispatcher(FakeSend()) is first
assert on_demand_dispatch.current() is first
finally:
on_demand_dispatch.reset_for_tests()
test/test_on_demand_mailbox.py
-106
View File
@@ -1,106 +0,0 @@
"""The on-demand request mailbox: how often it is read, and how it is consumed.
The mailbox is a cache key the web process writes and the display process
reads. Two properties matter and neither is obvious from the call site:
* it is polled after every rendered frame, so an uncached read here is a
disk read at frame rate;
* consuming it must not throw away a request that arrived while the previous
one was being processed.
"""
from unittest.mock import MagicMock
import pytest
class TestPollingIsBounded:
"""_poll_on_demand_requests runs ~125x/second on a scrolling mode.
The read is deliberately uncached (memory_ttl=0) because a cached one
pinned the first request for an hour. That makes the call a real disk read,
so it needs a floor -- without one it was ~125 reads per second to find
nothing at all.
"""
def test_first_call_always_reads(self, test_display_controller):
c = test_display_controller
c.cache_manager.get = MagicMock(return_value=None)
c._poll_on_demand_requests()
assert c.cache_manager.get.call_count == 1
def test_immediate_second_call_does_not_read(self, test_display_controller):
c = test_display_controller
c.cache_manager.get = MagicMock(return_value=None)
c._poll_on_demand_requests()
for _ in range(50):
c._poll_on_demand_requests()
assert c.cache_manager.get.call_count == 1, "polling was not bounded"
def test_reads_again_once_the_interval_has_passed(self, test_display_controller, monkeypatch):
c = test_display_controller
c.cache_manager.get = MagicMock(return_value=None)
clock = {"t": 1000.0}
monkeypatch.setattr("src.display_controller.time.monotonic", lambda: clock["t"])
c._poll_on_demand_requests()
clock["t"] += c.ON_DEMAND_POLL_INTERVAL / 2
c._poll_on_demand_requests()
assert c.cache_manager.get.call_count == 1, "read before the interval elapsed"
clock["t"] += c.ON_DEMAND_POLL_INTERVAL
c._poll_on_demand_requests()
assert c.cache_manager.get.call_count == 2
def test_the_interval_is_short_enough_to_feel_instant(self, test_display_controller):
# A person clicking in the web UI must not notice the floor.
assert test_display_controller.ON_DEMAND_POLL_INTERVAL <= 0.5
class TestMailboxIsConsumedByIdentity:
"""Deleting whatever is in the mailbox loses a request that raced in."""
def _arrange(self, controller, first, later):
"""Mailbox returns `first`, then `later` on the pre-delete re-read."""
controller.on_demand_active = False
controller.on_demand_request_id = None
controller._last_on_demand_poll = None
reads = iter([first, later])
def fake_get(key, *a, **kw):
if key == 'display_on_demand_request':
return next(reads, later)
return None # processed-id lookup
controller.cache_manager.get = MagicMock(side_effect=fake_get)
controller.cache_manager.set = MagicMock()
controller.cache_manager.delete = MagicMock()
controller._activate_on_demand = MagicMock()
REQ_A = {'request_id': 'A', 'action': 'start', 'plugin_id': 'p', 'mode': 'm'}
REQ_B = {'request_id': 'B', 'action': 'start', 'plugin_id': 'p', 'mode': 'm'}
def test_own_request_is_deleted(self, test_display_controller):
c = test_display_controller
self._arrange(c, self.REQ_A, self.REQ_A)
c._poll_on_demand_requests()
c.cache_manager.delete.assert_called_once_with('display_on_demand_request')
def test_a_newer_request_is_left_for_the_next_poll(self, test_display_controller):
c = test_display_controller
self._arrange(c, self.REQ_A, self.REQ_B)
c._poll_on_demand_requests()
assert c.cache_manager.delete.call_count == 0, \
"request B was deleted without ever being processed"
def test_an_already_empty_mailbox_is_still_cleared(self, test_display_controller):
c = test_display_controller
self._arrange(c, self.REQ_A, None)
c._poll_on_demand_requests()
c.cache_manager.delete.assert_called_once_with('display_on_demand_request')
def test_the_request_is_still_processed(self, test_display_controller):
c = test_display_controller
self._arrange(c, self.REQ_A, self.REQ_B)
c._poll_on_demand_requests()
c._activate_on_demand.assert_called_once()
test/test_on_demand_pinning_and_restart.py
+13 -38
View File
@@ -8,8 +8,9 @@ Three separate gaps, all reachable from the web UI's force-display dialog:
* restarting while on-demand was active loaded *only* the on-demand plugin,
so normal rotation had nothing to return to for the life of the process;
* a stop request was exempt from the duplicate guards on purpose and was
never removed from the mailbox, so it was re-processed on every poll
forever.
never removed from the file mailbox, so it was re-processed on every poll
forever. The mailbox is gone (stage 5); a stop still skips the guards,
so a second click stops a session a race left running.
"""
from unittest.mock import MagicMock
@@ -185,51 +186,25 @@ class TestRestartDoesNotStarveTheOtherPlugins:
assert controller.on_demand_active is False
class TestStopRequestsAreConsumed:
"""A stop request is exempt from the duplicate guards, so the mailbox
delete is the only thing that ends it."""
STOP = {'request_id': 'S1', 'action': 'stop'}
class TestStopRequestsSkipTheDuplicateGuard:
"""A stop is exempt from the request-id guard: every one is acted on."""
def _arrange(self, controller, active):
controller.on_demand_active = active
controller.on_demand_status = 'active' if active else 'idle'
controller._last_on_demand_poll = None
controller.cache_manager.get = MagicMock(
side_effect=lambda key, *a, **kw:
self.STOP if key == 'display_on_demand_request' else None)
controller.cache_manager.set = MagicMock()
controller.cache_manager.delete = MagicMock()
controller._clear_on_demand = MagicMock()
def test_a_handled_stop_is_removed_from_the_mailbox(self, test_display_controller):
def test_the_stop_is_acted_on(self, test_display_controller):
c = test_display_controller
self._arrange(c, active=True)
c._poll_on_demand_requests()
c.cache_manager.delete.assert_called_once_with('display_on_demand_request')
def test_a_stop_arriving_while_idle_is_also_removed(self, test_display_controller):
"""Otherwise a stop sent to an idle display re-fires forever."""
c = test_display_controller
self._arrange(c, active=False)
c._poll_on_demand_requests()
c.cache_manager.delete.assert_called_once_with('display_on_demand_request')
def test_the_stop_is_still_acted_on(self, test_display_controller):
c = test_display_controller
self._arrange(c, active=True)
c._poll_on_demand_requests()
c._handle_on_demand_request({'request_id': 'S1', 'action': 'stop',
'source': 'socket'})
c._clear_on_demand.assert_called_once_with(reason='requested-stop')
def test_a_start_racing_in_behind_a_stop_is_not_discarded(self, test_display_controller):
"""The compare-before-delete applies to stops too."""
def test_the_same_stop_twice_is_acted_on_twice(self, test_display_controller):
c = test_display_controller
self._arrange(c, active=True)
newer = {'request_id': 'S2', 'action': 'start', 'plugin_id': 'p', 'mode': 'm'}
reads = iter([self.STOP, newer])
c.cache_manager.get = MagicMock(
side_effect=lambda key, *a, **kw:
next(reads, newer) if key == 'display_on_demand_request' else None)
c._poll_on_demand_requests()
assert c.cache_manager.delete.call_count == 0
for _ in range(2):
c._handle_on_demand_request({'request_id': 'S1', 'action': 'stop',
'source': 'socket'})
assert c._clear_on_demand.call_count == 2
test/test_plugin_on_demand_api.py
+38 -50
View File
@@ -2,19 +2,17 @@
and end_on_demand().
A plugin running in the display process used to write the
``display_on_demand_request`` mailbox, which the display reads once a second
while the control socket is up. These tests pin the way in that replaces it:
``display_on_demand_request`` mailbox, which the display no longer reads
(stage 5). These tests pin the way in that replaced it:
* BasePlugin -> PluginManager -> DisplayController.submit_plugin_on_demand,
which only queues, from any thread;
* the render thread applies the queue where it applies socket commands,
through the mailbox's own handler, without the mailbox's read floor, and
woken by the control socket when it is up;
through _handle_on_demand_request, at once, and woken by the control
socket when it is up;
* a plugin's stop ends only its own session;
* no display to ask (the web interface's plugin manager, an old core's
plugin manager) answers None, which is a plugin's cue to fall back to the
mailbox;
* the mailbox still works for plugins that write it.
plugin manager) answers None.
"""
import logging
@@ -73,19 +71,10 @@ def controller(test_display_controller):
c_ = test_display_controller
c_.on_demand_active = False
c_.on_demand_request_id = None
c_._last_on_demand_poll = None
mailbox = {'value': None}
def fake_get(key, *a, **kw):
if key == 'display_on_demand_request':
return mailbox['value']
return None
c_.cache_manager.get = MagicMock(side_effect=fake_get)
c_.cache_manager.get = MagicMock(return_value=None)
c_.cache_manager.set = MagicMock()
c_.cache_manager.delete = MagicMock()
c_._activate_on_demand = MagicMock()
c_.mailbox = mailbox
return c_
@@ -101,7 +90,7 @@ class TestWiring:
controller.plugin_manager.set_on_demand_handler.assert_called_once_with(
controller.submit_plugin_on_demand)
def test_a_start_reaches_the_mailbox_handler(self, wired):
def test_a_start_reaches_the_on_demand_handler(self, wired):
controller, manager = wired
rid = _plugin('pomodoro-timer', manager).request_on_demand(
mode='pomodoro', duration=30, pinned=True)
@@ -139,15 +128,6 @@ class TestWiring:
controller._poll_on_demand_requests()
assert seen == ['a', 'b', 'c']
def test_the_mailbox_still_works_for_older_plugins(self, wired):
controller, manager = wired
controller.mailbox['value'] = {'request_id': 'mb', 'action': 'start',
'plugin_id': 'birdnet-go'}
_plugin('on-air', manager).request_on_demand()
controller._poll_on_demand_requests()
ids = [call.args[0]['request_id'] for call in controller._activate_on_demand.call_args_list]
assert 'mb' in ids and len(ids) == 2
def test_a_failing_request_is_contained(self, wired):
controller, manager = wired
calls = []
@@ -174,10 +154,10 @@ class TestPromptness:
controller._service_pending_changes() # well inside the 0.25 s floor
controller._activate_on_demand.assert_called_once()
def test_a_plugin_request_skips_the_mailbox_floor(self, wired):
def test_a_plugin_request_lands_on_the_next_poll(self, wired):
controller, manager = wired
controller._control_server = _WakeServer()
controller._poll_on_demand_requests() # sets the 1 s mailbox floor
controller._poll_on_demand_requests()
_plugin('p', manager).request_on_demand()
controller._poll_on_demand_requests()
controller._activate_on_demand.assert_called_once()
@@ -286,14 +266,13 @@ class TestStop:
controller._poll_on_demand_requests()
controller._clear_on_demand.assert_not_called()
def test_a_mailbox_stop_still_ends_any_session(self, wired):
def test_a_socket_stop_still_ends_any_session(self, wired):
controller, _ = wired
controller.on_demand_active = True
controller.on_demand_plugin_id = 'clock'
controller._clear_on_demand = MagicMock()
controller.mailbox['value'] = {'request_id': 's', 'action': 'stop',
'plugin_id': 'on-air'}
controller._poll_on_demand_requests()
controller._handle_on_demand_request({'request_id': 's', 'action': 'stop',
'source': 'socket'})
controller._clear_on_demand.assert_called_once_with(reason='requested-stop')
def test_start_then_stop_from_one_thread_ends_the_session(self, wired):
@@ -314,7 +293,7 @@ class TestStop:
class TestNoDisplay:
"""None is a plugin's cue to write the mailbox instead."""
"""None: no display in this process took the request."""
def test_a_manager_with_no_handler_answers_none(self):
plugin = _plugin('p', _manager())
@@ -353,21 +332,30 @@ class TestNoDisplay:
assert bare._plugin_on_demand_pending() is False
bare._drain_plugin_on_demand() # nothing to do, no error
def test_the_feature_detection_pattern(self):
"""The hasattr pattern from docs/PLUGIN_API_REFERENCE.md."""
writes = []
class OldCorePlugin: # an older core's BasePlugin has no such method
pass
for plugin, expect_mailbox in ((OldCorePlugin(), True),
(_plugin('p', _manager()), True),
(_plugin('p', _manager(lambda r: True)), False)):
writes.clear()
if not (hasattr(plugin, 'request_on_demand')
and plugin.request_on_demand(mode='m')):
writes.append('mailbox')
assert (writes == ['mailbox']) is expect_mailbox
def test_a_mailbox_write_after_none_is_dropped_with_a_warning(self, tmp_path,
monkeypatch, caplog):
"""What a plugin written for older cores does on None now: its
fallback write to the retired key stores nothing, and the log names
it once."""
from src import cache_manager as cache_module
from src.cache_manager import CacheManager
monkeypatch.setattr(CacheManager, '_get_writable_cache_dir',
lambda self: str(tmp_path))
monkeypatch.setattr(cache_module, '_retired_writers_warned', set())
cache = CacheManager()
try:
plugin = _plugin('birdnet-go', _manager())
plugin.cache_manager = cache
caplog.set_level(logging.WARNING)
for _ in range(2):
if plugin.request_on_demand(mode='m') is None:
plugin.cache_manager.set('display_on_demand_request', {
'request_id': 'r', 'action': 'start', 'plugin_id': 'birdnet-go'})
assert cache.get('display_on_demand_request', max_age=None, memory_ttl=0) is None
lines = [r.getMessage() for r in caplog.records if 'retired' in r.getMessage()]
assert len(lines) == 1 and "plugin 'birdnet-go'" in lines[0]
finally:
cache.stop_cleanup_thread()
class TestArguments:
@@ -401,7 +389,7 @@ class TestArguments:
class TestMockManagers:
def test_a_magicmock_manager_reads_as_not_taken(self):
"""A plugin's test with a MagicMock manager keeps its mailbox path."""
"""A plugin's test with a MagicMock manager reads as "not taken"."""
plugin = _plugin('p', MagicMock())
assert plugin.request_on_demand(mode='m') is None
assert plugin.end_on_demand() is None
test/test_run_loop_socket_wake.py
+10 -19
View File
@@ -12,8 +12,8 @@ frame, so a command is applied:
* at the next frame in Vegas (8 ms here, at 125 fps).
Each test runs the real run() loop on the fake clock of
test/_run_loop_harness.py and compares the socket with the mailbox for the
same request. The latencies are the fake clock's, so they are exact.
test/_run_loop_harness.py. The latencies are the fake clock's, so they are
exact. (The file mailbox these were once compared with is gone: stage 5.)
"""
import os
@@ -32,13 +32,10 @@ def _event_time(trace, kind):
return next(e[0] for e in trace["events"] if e[1] == kind)
def _on_demand_latency(tmp_path, build, via, horizon=40):
def _on_demand_latency(tmp_path, build, horizon=40):
h = RunLoopHarness(tmp_path, horizon=horizon)
build(h)
if via == "socket":
h.control_socket().post(POSTED, Command.ON_DEMAND_START, {"plugin_id": "weather"})
else:
h.on_demand_request(POSTED, "mb1", plugin_id="weather")
h.control_socket().post(POSTED, Command.ON_DEMAND_START, {"plugin_id": "weather"})
trace = h.run()
return round(_event_time(trace, "on-demand-start") - POSTED, 3), trace
@@ -60,18 +57,12 @@ def _vegas(h):
h.enable_vegas(cycle=30)
@pytest.mark.parametrize("build, mailbox_latency", [
(_static, 0.7), # the next 1 s frame, at t=11
(_dwell, 0.2), # the next 0.25 s tick, at t=10.5
(_vegas, 0.02), # the next 10-frame check: 80 ms at 125 fps, ~0.4 s on a Pi 4
], ids=["static-screen", "dwell", "vegas"])
def test_a_socket_command_lands_at_once(tmp_path, build, mailbox_latency):
(tmp_path / "s").mkdir()
(tmp_path / "m").mkdir()
socket_latency, trace = _on_demand_latency(tmp_path / "s", build, "socket")
via_mailbox, _ = _on_demand_latency(tmp_path / "m", build, "mailbox")
assert via_mailbox == pytest.approx(mailbox_latency, abs=0.002)
@pytest.mark.parametrize("build", [_static, _dwell, _vegas],
ids=["static-screen", "dwell", "vegas"])
def test_a_socket_command_lands_at_once(tmp_path, build):
# Before stage 2 these waited for the next 1 s frame, the next 0.25 s
# dwell tick, or the next 10-frame Vegas check.
socket_latency, trace = _on_demand_latency(tmp_path, build)
if build is _vegas:
# One 8 ms frame: Vegas checks the queue every frame now.
assert socket_latency <= 0.008
test/test_web_api.py
+11 -8
View File
@@ -528,14 +528,17 @@ class TestDisplayAPI:
from web_interface.blueprints.api_v3 import api_v3
mock_cache_manager = api_v3.cache_manager = MagicMock()
response = client.post('/api/v3/display/on-demand/stop')
# May return 200 if successful or 500 on error
assert response.status_code in [200, 500]
# Verify stop request was set in cache if successful
if response.status_code == 200:
assert mock_cache_manager.set.called
with patch('web_interface.blueprints.api_v3.display.control_client.on_demand_stop',
side_effect=lambda request_id, **kw: {'accepted': True}) as stop:
response = client.post('/api/v3/display/on-demand/stop')
assert response.status_code == 200
assert response.get_json()['data']['transport'] == 'socket'
stop.assert_called_once()
# The request goes over the control socket; nothing is written to
# the cache (the file mailbox is gone).
mock_cache_manager.set.assert_not_called()
class TestPluginsAPI:
test/web_interface/test_api_v3_unhandled_errors.py
-1
View File
@@ -81,7 +81,6 @@ REMOVED_CATCH_ALLS = [
("GET", "/api/v3/config/main", None),
("GET", "/api/v3/config/secrets", None),
("GET", "/api/v3/display/modes", None),
("POST", "/api/v3/display/on-demand/stop", {}),
("GET", "/api/v3/cache/list", None),
("GET", "/api/v3/plugins/installed", None),
("GET", "/api/v3/plugins/health", None),