mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 22:35:08 +00:00
fix(web): uniqueItems saves, /health count, Vegas order wipe; one list-repair helper (#638)
* fix(web): drop repeats from uniqueItems lists before validating a plugin save dedup_unique_arrays lost its only caller in #330, so submitting a value a uniqueItems list already holds (a stock symbol saved once and posted again) failed the whole save with a validation error. _prepare_plugin_config_for_save runs it again just before validation, which covers both POST /plugins/config and plugin sections posted to /config/main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): /health counts the discovered plugins and logs the checks it fails The plugin check counted plugin_manager.get_available_plugins(), which PluginManager does not have, behind a hasattr guard that made plugin_count 0 on every device. It now counts the discovered manifests, discovering first when nothing has been scanned yet. The config, plugin and hardware checks answered "see logs for details" without logging anything. Each now logs a warning with the traceback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): store refresh no longer claims a commit-metadata refresh POST /plugins/store/refresh read fetch_commit_info (or fetch_latest_versions) only to append "(with refreshed commit metadata from GitHub)" to its message. It never fetched any: the route re-downloads the registry and nothing else. search_plugins takes the flag, but it reads commit info through its cache, so passing it on would not refresh anything either. The flag is ignored now and the message says what happened. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): refuse a malformed Vegas plugin order instead of clearing it A vegas_plugin_order or vegas_excluded_plugins value that was not JSON, or not a list, was stored as [] and the save answered 200, so a bad value wiped the saved order or exclusions. Both now answer 400 and save nothing, the way plugin_rotation_order already did; the three share one parser. A list that holds anything but plugin-id strings is refused as well. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): per-plugin health and metrics read the display service's latest GET /plugins/health/<id> and /plugins/metrics/<id> called get_health_summary and get_metrics_summary without force_reload, so they answered with whatever the web process read first and kept in memory, while the display service kept writing newer state. They now pass force_reload=True, as the list routes do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): plugin config reset saves through the shared atomic save POST /plugins/config/reset called config_manager.save_config directly, so it took no backup, and a failed write escaped as an unhandled exception. It then handed on_config_change the raw stored section, not the prepared config a loaded plugin runs with. It now saves through _save_config_atomic with a backup, answers CONFIG_SAVE_FAILED when that fails, and notifies with _prepared_plugin_config, as POST /plugins/config does. POST /plugins/toggle carried its own copy of _save_config_atomic's save_config_atomic-or-save_config fallback; it calls the shared helper now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): one reading and one "unavailable" for each system metric system_metrics.collect_system_metrics() promised None for a metric it could not read, but returned cpu_temp as 0 off a Pi, and the whole no-psutil fallback as zeros. GET /system/status measured the same numbers a second time with its own code, and answered None there. Now both come from collect_system_metrics(), and "unavailable" is None everywhere. /system/status keeps its 0.1s CPU sample and its 10s cache, and gains nothing it did not already send. Two differences: without psutil it answers 200 with null metrics instead of 503, and a disk it cannot stat is null instead of a 500. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): /display/current sends the snapshot as-is and logs a failed read GET /display/current PIL-decoded the preview snapshot and re-encoded it before base64-ing it, spending CPU on the Pi to send the same picture, and dropped any failure with `except Exception: pass`. The /stream/display SSE stream already passed the PNG's bytes straight through. Both now read through web_interface/display_preview.py and answer with the same payload. A missing snapshot is still a null image; any other read failure is logged as a warning. /health reads the snapshot path from the same module. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): one helper puts a submitted plugin config's lists back The plugin-config save turned position-keyed dicts ({"0": ..., "1": ...}) back into lists in five copies: four in the form path's fix_array_structures (whose prefix branches never ran, since no caller passed one), and _fix_json_arrays on the JSON path. It then force-fixed the news plugin's feeds.custom_feeds by name, in case the generic pass had missed it. src/web_interface/config_arrays.coerce_array_shapes now does it for both paths, custom_feeds included. ensure_array_defaults duplicated _fix_none_arrays and is gone. In the same function: the union-type re-checks that the null handling above them made unreachable, the "(temporary)" random_seed debug log, and a commented-out log line are removed. A failed validation is logged once as a warning, not four ERROR lines and a WARNING. Element types are left to normalize_config_values, which already converted them for both paths. One difference: the form path no longer adds an empty {} for a nested object the post left out that has no defaults. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): import at module top and log through the module logger The web_interface.cache imports in config.py and fonts.py were wrapped in `except ImportError` fallbacks. It is an in-repo module that imports nothing from the project, so it cannot fail to import; it is imported once at module top, as system.py now does. cache.py's docstring said blueprints import it lazily "to avoid circular imports"; it now says why that is unnecessary. Five logging.error calls in the dim-schedule GET and three logging.warning calls in plugins.py went to the root logger; they use the module logger. Function-local re-imports of json, os, shutil, logging and Path, all already imported by the module, are gone. The `import os` inside two except blocks of save_plugin_config also made os a local name for the whole function. execute_plugin_action's step-1 handler gets a comment saying why it stays: it looks like a copy of the blueprint handler, but without it a TimeoutExpired from the plugin's script would reach the route's own `except subprocess.TimeoutExpired` and be answered as a 408. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): app.py loses dead CSRF and reconciliation state, comments fixed - csrf was always None, so `if csrf: csrf.exempt(...)` never ran, and its note that the api_v3 blueprint "is exempted above" named an exemption that does not exist. Both are gone; the reason there is no CSRF protection stays, shortened. - The SSE rate-limit comment called the default "tight" at 20 per minute. The default is 1000 per minute and the streams' 200 is the tighter one; the comment now says so. The limits are unchanged. - _reconciliation_done was written and never read. The docstring that explains why reconciliation runs once keeps its reason, in the present tense. - Removed: a dangling "import cache functions" comment with no import under it, a "security check ... within project_root" label on an existence check, the "(simplified version)" narration, and the note that no redirect route is needed. The preview loop's sleep comment no longer mentions a PIL encode that the loop does not do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(web): api_v3 comments name the package __init__, not a _common module Every route module's docstring said the shared blueprint comes "from ._common", a module the package split never created; they name the package __init__. The PROJECT_ROOT comment described the path from _common.py; it now describes this package and keeps the incident it guards against. The "(corrected) in this commit" note in resolve_pull_command and the /health comment the split's mechanical time -> _pkg.time rewrite garbled ("Stamp the start _pkg.time") read correctly again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): drop hasattr checks for attributes PluginManager always has PluginManager.__init__ sets health_tracker and resource_monitor (to None until they are configured), so the seven hasattr(api_v3.plugin_manager, ...) guards in the health, metrics and limits routes were always true. The falsy checks that do the work stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): pages_v3 dispatches partials from a dict with one error handler load_partial chose a loader through a fourteen-branch if/elif, and thirteen of the loaders then wrapped themselves in the same try/except, logging "Error loading partial" without saying which. The route now looks the name up in _PARTIAL_LOADERS and has the one handler, which logs the partial's name. The loaders just render. _load_tools_partial keeps its own messages. The search index's _partial_html already catches a loader that raises. serve_plugin_web_ui repeated _plugin_dir_for inline (containment plus the ledmatrix- prefix fallback); it calls it now. Also removed: the unused markupsafe.escape import, function-local json/Path re-imports, and unused exception bindings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): remove unused imports, locals and a try that cannot fail - get_error_aggregator was imported by the api_v3 package and used by no one; seven names config.py imported, and Path in misc.py and logging in plugins.py, likewise. - branch_info in install_plugin was built and never logged; test_config in /health was bound and never read (the load_config call is the check). - An f-string with no placeholders in the asset upload route. - _installed_plugin_ids wrapped list(manifests.keys()) in try/except; _discovered_plugin_manifests always returns a dict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): start.py logs its startup lines and drops unreachable branches The startup banner went to stdout with print(); it goes through a logger now, which the app import has already configured, so it reaches the journal with a level and timestamp like every other line. The "no addresses" branch is gone: get_local_ips() always returns at least "localhost". The except around app.run re-raised "only if it's not a client disconnection error" from inside the branch that had just established it was one, so that raise could not run. It is one check now, on a named tuple of the errnos, which the werkzeug log filter uses too. The comment on threaded=True counts three SSE endpoints, which is how many there are. Trailing whitespace is stripped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): save_main_config names its General fields once The General tab's field names were listed twice, once to detect a General form post and again, with four more, to keep the remaining-keys merge from storing them as top-level keys. GENERAL_FIELDS and _MAPPED_TOP_LEVEL_FIELDS hold them now, and the four per-section skip checks are one set. The comment on that merge said plugin configs are handled "here too", and "(including plugin keys)". Plugin sections are handled and removed from the body before it runs; the comment says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): plugin directories come from the plugin manager only Six lookups fell back to PROJECT_ROOT/plugins/<id> when there was no plugin manager: GET /plugins/config's of-the-day data, POST /plugins/action, the plugin static-file route, the calendar credentials upload and the calendar OAuth routes. The loader never scans plugins/ (PluginManager.discover_plugins reads only the configured directory, plugin-repos by default), so what they found there was a plugin that never runs. _plugin_directory() asks the manager and answers None without one, which each route already reports as "not found". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): web-backend Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
@@ -0,0 +1,90 @@
|
||||
"""GET /api/v3/health: the plugin count is real, and a failed check is logged.
|
||||
|
||||
The plugin check counted ``plugin_manager.get_available_plugins()``, which
|
||||
PluginManager does not have; a hasattr guard turned that into a permanent 0.
|
||||
Each check that fails answers "see logs for details", so it has to log.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
|
||||
|
||||
URL = "/api/v3/health"
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _no_systemctl(monkeypatch):
|
||||
monkeypatch.setattr("web_interface.blueprints.api_v3.misc._get_display_service_status",
|
||||
lambda: {"active": True})
|
||||
|
||||
|
||||
def _checks(client):
|
||||
response = client.get(URL)
|
||||
assert response.status_code == 200, response.get_json()
|
||||
return response.get_json()["data"]["checks"]
|
||||
|
||||
|
||||
def test_plugin_count_is_the_number_of_discovered_plugins(api_v3_client, api_v3_module):
|
||||
api_v3_module.api_v3.plugin_manager.plugin_manifests = {
|
||||
"clock": {"id": "clock"}, "weather": {"id": "weather"}, "stocks": {"id": "stocks"},
|
||||
}
|
||||
|
||||
check = _checks(api_v3_client)["plugin_system"]
|
||||
|
||||
assert check == {"status": "operational", "plugin_count": 3}
|
||||
|
||||
|
||||
def test_plugin_count_discovers_when_nothing_is_discovered_yet(api_v3_client, api_v3_module):
|
||||
pm = api_v3_module.api_v3.plugin_manager
|
||||
pm.plugin_manifests = {}
|
||||
|
||||
def discover():
|
||||
pm.plugin_manifests = {"clock": {"id": "clock"}}
|
||||
pm.discover_plugins.side_effect = discover
|
||||
|
||||
assert _checks(api_v3_client)["plugin_system"]["plugin_count"] == 1
|
||||
|
||||
|
||||
def test_a_failed_config_check_is_logged(api_v3_client, api_v3_module, caplog):
|
||||
api_v3_module.api_v3.config_manager.load_config.side_effect = OSError("disk gone")
|
||||
|
||||
with caplog.at_level(logging.WARNING):
|
||||
check = _checks(api_v3_client)["config_file"]
|
||||
|
||||
assert check["error"] == "see logs for details"
|
||||
logged = [r for r in caplog.records if "config file" in r.getMessage()]
|
||||
assert logged and logged[0].exc_info and "disk gone" in str(logged[0].exc_info[1])
|
||||
|
||||
|
||||
def test_a_failed_plugin_check_is_logged(api_v3_client, api_v3_module, caplog, monkeypatch):
|
||||
def boom():
|
||||
raise RuntimeError("manifests unreadable")
|
||||
monkeypatch.setattr("web_interface.blueprints.api_v3.misc._discovered_plugin_manifests", boom)
|
||||
|
||||
with caplog.at_level(logging.WARNING):
|
||||
check = _checks(api_v3_client)["plugin_system"]
|
||||
|
||||
assert check["status"] == "error"
|
||||
logged = [r for r in caplog.records if "count plugins" in r.getMessage()]
|
||||
assert logged and logged[0].exc_info
|
||||
|
||||
|
||||
def test_a_failed_hardware_check_is_logged(api_v3_client, api_v3_module, caplog, monkeypatch):
|
||||
def getmtime(_path):
|
||||
raise PermissionError("denied")
|
||||
fake_os = SimpleNamespace(path=SimpleNamespace(exists=lambda _p: True, getmtime=getmtime))
|
||||
monkeypatch.setattr("web_interface.blueprints.api_v3.misc.os", fake_os)
|
||||
|
||||
with caplog.at_level(logging.WARNING):
|
||||
check = _checks(api_v3_client)["hardware"]
|
||||
|
||||
assert check["status"] == "unknown"
|
||||
logged = [r for r in caplog.records if "snapshot" in r.getMessage()]
|
||||
assert logged and logged[0].exc_info
|
||||
@@ -0,0 +1,68 @@
|
||||
"""GET /plugins/health/<id> and /plugins/metrics/<id> read the display
|
||||
service's latest state, not the web process's first snapshot.
|
||||
|
||||
The display service writes health and metrics to the shared cache; the web
|
||||
process only reads them. Its tracker and monitor keep what they read first in
|
||||
memory, so without ``force_reload`` the per-plugin routes kept answering with
|
||||
that first read while the list routes (which pass it) moved on.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from src.plugin_system.plugin_health import PluginHealthTracker # noqa: E402
|
||||
from src.plugin_system.resource_monitor import PluginResourceMonitor # noqa: E402
|
||||
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
|
||||
|
||||
|
||||
class SharedCache:
|
||||
"""The on-disk cache both processes see, reduced to a dict."""
|
||||
|
||||
def __init__(self):
|
||||
self.entries = {}
|
||||
|
||||
def get(self, key, max_age=None, memory_ttl=None):
|
||||
return self.entries.get(key)
|
||||
|
||||
def set(self, key, value, *args, **kwargs):
|
||||
self.entries[key] = value
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def shared_cache(api_v3_module):
|
||||
cache = SharedCache()
|
||||
pm = api_v3_module.api_v3.plugin_manager
|
||||
pm.health_tracker = PluginHealthTracker(cache)
|
||||
pm.resource_monitor = PluginResourceMonitor(cache)
|
||||
return cache
|
||||
|
||||
|
||||
def test_health_reflects_failures_recorded_after_the_first_read(api_v3_client, shared_cache):
|
||||
first = api_v3_client.get("/api/v3/plugins/health/weather").get_json()["data"]
|
||||
assert first["total_failures"] == 0
|
||||
|
||||
display_side = PluginHealthTracker(shared_cache)
|
||||
display_side.record_failure("weather", RuntimeError("api down"))
|
||||
display_side.record_failure("weather", RuntimeError("api down"))
|
||||
|
||||
later = api_v3_client.get("/api/v3/plugins/health/weather").get_json()["data"]
|
||||
assert later["total_failures"] == 2
|
||||
|
||||
|
||||
def test_metrics_reflect_calls_recorded_after_the_first_read(api_v3_client, shared_cache):
|
||||
first = api_v3_client.get("/api/v3/plugins/metrics/weather").get_json()["data"]
|
||||
assert first["call_count"] == 0
|
||||
|
||||
shared_cache.set("plugin_metrics:weather", {
|
||||
"memory_mb": 12.5, "cpu_percent": 3.0, "execution_time": 0.2,
|
||||
"call_count": 40, "total_execution_time": 8.0,
|
||||
"max_execution_time": 0.5, "min_execution_time": 0.1,
|
||||
"last_update_time": 1000.0,
|
||||
})
|
||||
|
||||
later = api_v3_client.get("/api/v3/plugins/metrics/weather").get_json()["data"]
|
||||
assert later["call_count"] == 40
|
||||
@@ -66,12 +66,16 @@ class TestRefreshPluginStore:
|
||||
assert response.status_code == 200
|
||||
|
||||
@pytest.mark.parametrize("key", ["fetch_commit_info", "fetch_latest_versions"])
|
||||
def test_either_commit_info_key_extends_the_message(
|
||||
def test_commit_info_flag_claims_no_refresh_it_does_not_do(
|
||||
self, api_v3_client, api_v3_module, key):
|
||||
# fetch_latest_versions is the older spelling; both must work.
|
||||
api_v3_module.api_v3.plugin_store_manager.fetch_registry.return_value = {"plugins": []}
|
||||
# The route only re-downloads the registry. It used to append "(with
|
||||
# refreshed commit metadata from GitHub)" for either flag without
|
||||
# fetching any.
|
||||
store = api_v3_module.api_v3.plugin_store_manager
|
||||
store.fetch_registry.return_value = {"plugins": [{"id": "a"}]}
|
||||
response = api_v3_client.post(self.URL, json={key: True})
|
||||
assert "commit metadata" in response.get_json()["message"]
|
||||
assert response.get_json()["message"] == "Plugin store refreshed"
|
||||
store.fetch_registry.assert_called_once_with(force_refresh=True)
|
||||
|
||||
def test_message_stays_plain_without_the_flag(self, api_v3_client, api_v3_module):
|
||||
api_v3_module.api_v3.plugin_store_manager.fetch_registry.return_value = {"plugins": []}
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
"""POST /config/main refuses a malformed Vegas plugin order or exclusion list.
|
||||
|
||||
Both were parsed with ``except JSONDecodeError: ... = []``, so a bad value
|
||||
cleared the saved list and answered 200. They now fail the save with a 400,
|
||||
as plugin_rotation_order already did.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
|
||||
|
||||
URL = "/api/v3/config/main"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def saved(api_v3_module):
|
||||
config = {"display": {"vegas_scroll": {"plugin_order": ["clock", "weather"],
|
||||
"excluded_plugins": ["stocks"]}}}
|
||||
cm = api_v3_module.api_v3.config_manager
|
||||
cm.load_config.return_value = config
|
||||
cm.save_config_atomic.return_value.status.value = 'success'
|
||||
return cm
|
||||
|
||||
|
||||
@pytest.mark.parametrize("field", ["vegas_plugin_order", "vegas_excluded_plugins"])
|
||||
@pytest.mark.parametrize("value", ["[not json", '{"a": 1}', "[1, 2]", 7])
|
||||
def test_malformed_list_is_refused_and_nothing_is_saved(api_v3_client, saved, field, value):
|
||||
response = api_v3_client.post(URL, json={field: value})
|
||||
|
||||
assert response.status_code == 400, response.get_json()
|
||||
assert field in response.get_json()["message"]
|
||||
saved.save_config_atomic.assert_not_called()
|
||||
saved.save_config.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ['["weather", "clock"]', ["weather", "clock"]])
|
||||
def test_json_text_or_array_is_stored(api_v3_client, saved, value):
|
||||
response = api_v3_client.post(URL, json={"vegas_plugin_order": value})
|
||||
|
||||
assert response.status_code == 200, response.get_json()
|
||||
stored = saved.save_config_atomic.call_args.args[0]
|
||||
assert stored["display"]["vegas_scroll"]["plugin_order"] == ["weather", "clock"]
|
||||
assert stored["display"]["vegas_scroll"]["excluded_plugins"] == ["stocks"]
|
||||
|
||||
|
||||
def test_rotation_order_keeps_its_messages(api_v3_client, saved):
|
||||
response = api_v3_client.post(URL, json={"plugin_rotation_order": "[oops"})
|
||||
|
||||
assert response.status_code == 400
|
||||
assert response.get_json()["message"] == "plugin_rotation_order must be valid JSON"
|
||||
@@ -0,0 +1,79 @@
|
||||
"""GET /api/v3/display/current passes the snapshot PNG through untouched.
|
||||
|
||||
It used to PIL-decode the snapshot and re-encode it, which cost CPU on the Pi
|
||||
for no change in the picture, and it swallowed any read failure with
|
||||
``except Exception: pass``. It now sends the file's own bytes, the payload the
|
||||
/stream/display SSE stream sends, and logs a failed read.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import io
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
from flask import Flask
|
||||
from PIL import Image, PngImagePlugin
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from web_interface import display_preview # noqa: E402
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client(monkeypatch):
|
||||
from web_interface.blueprints.api_v3 import api_v3
|
||||
monkeypatch.setattr(api_v3, 'config_manager', MagicMock(), raising=False)
|
||||
api_v3.config_manager.load_config.return_value = {}
|
||||
app = Flask(__name__)
|
||||
app.config['TESTING'] = True
|
||||
app.register_blueprint(api_v3, url_prefix='/api/v3')
|
||||
with app.test_client() as test_client:
|
||||
yield test_client
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def snapshot(tmp_path, monkeypatch):
|
||||
"""A snapshot PNG carrying a text chunk, which a PIL re-encode drops."""
|
||||
info = PngImagePlugin.PngInfo()
|
||||
info.add_text('written-by', 'display_manager')
|
||||
buffer = io.BytesIO()
|
||||
Image.new('RGB', (4, 2), (255, 0, 0)).save(buffer, format='PNG', pnginfo=info)
|
||||
path = tmp_path / 'led_matrix_preview.png'
|
||||
path.write_bytes(buffer.getvalue())
|
||||
monkeypatch.setattr(display_preview, 'SNAPSHOT_PATH', str(path))
|
||||
return path
|
||||
|
||||
|
||||
def _image(client):
|
||||
response = client.get('/api/v3/display/current')
|
||||
assert response.status_code == 200
|
||||
return response.get_json()['data']
|
||||
|
||||
|
||||
def test_the_snapshot_bytes_are_sent_as_they_are(client, snapshot):
|
||||
data = _image(client)
|
||||
assert base64.b64decode(data['image']) == snapshot.read_bytes()
|
||||
|
||||
|
||||
def test_route_and_stream_send_the_same_payload_keys(client, snapshot):
|
||||
data = _image(client)
|
||||
assert set(data) == set(display_preview.preview_payload(1, 1, None))
|
||||
|
||||
|
||||
def test_no_snapshot_is_a_null_image_without_a_warning(client, tmp_path, monkeypatch, caplog):
|
||||
monkeypatch.setattr(display_preview, 'SNAPSHOT_PATH', str(tmp_path / 'missing.png'))
|
||||
with caplog.at_level(logging.WARNING):
|
||||
assert _image(client)['image'] is None
|
||||
assert not [r for r in caplog.records if 'snapshot' in r.getMessage()]
|
||||
|
||||
|
||||
def test_an_unreadable_snapshot_is_logged(client, snapshot, monkeypatch, caplog):
|
||||
def denied(_path=None):
|
||||
raise PermissionError('denied')
|
||||
monkeypatch.setattr(display_preview, 'read_snapshot_base64', denied)
|
||||
with caplog.at_level(logging.WARNING):
|
||||
assert _image(client)['image'] is None
|
||||
assert [r for r in caplog.records if 'snapshot' in r.getMessage() and r.exc_info]
|
||||
@@ -31,8 +31,12 @@ import pytest
|
||||
PROJECT_ROOT = Path(__file__).parent.parent
|
||||
sys.path.insert(0, str(PROJECT_ROOT))
|
||||
|
||||
from web_interface import system_metrics # noqa: E402
|
||||
from web_interface.system_metrics import collect_system_metrics # noqa: E402
|
||||
|
||||
MB = 1024 * 1024
|
||||
GB = 1024 * MB
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def metrics(monkeypatch):
|
||||
@@ -45,10 +49,11 @@ def metrics(monkeypatch):
|
||||
lambda _p: (_ for _ in ()).throw(OSError("no such mount")))
|
||||
else:
|
||||
monkeypatch.setattr(psutil, "disk_usage",
|
||||
lambda _p: SimpleNamespace(percent=disk_percent))
|
||||
lambda _p: SimpleNamespace(percent=disk_percent,
|
||||
total=32 * GB, used=4 * GB))
|
||||
monkeypatch.setattr(psutil, "virtual_memory",
|
||||
lambda: SimpleNamespace(percent=used_percent,
|
||||
available=available_bytes))
|
||||
lambda: SimpleNamespace(percent=used_percent, total=1024 * MB,
|
||||
used=600 * MB, available=available_bytes))
|
||||
return collect_system_metrics()
|
||||
return _collect
|
||||
|
||||
@@ -101,3 +106,18 @@ class TestEveryPredictiveFieldIsPresent:
|
||||
"memory_available_mb", "disk_used_percent"):
|
||||
assert field in m
|
||||
assert m["disk_used_percent"] is None
|
||||
assert all(m[key] is None for key in m if key != "cpu_temp")
|
||||
|
||||
|
||||
class TestUnavailableIsNull:
|
||||
"""One answer for "could not be read": None, never 0."""
|
||||
|
||||
def test_unreadable_temperature_is_null_not_zero_degrees(self, metrics, monkeypatch):
|
||||
monkeypatch.setattr(system_metrics, "_THERMAL_ZONE", "/nonexistent/thermal/temp")
|
||||
assert metrics()["cpu_temp"] is None
|
||||
|
||||
def test_readable_temperature_is_degrees_c(self, metrics, monkeypatch, tmp_path):
|
||||
zone = tmp_path / "temp"
|
||||
zone.write_text("48312\n")
|
||||
monkeypatch.setattr(system_metrics, "_THERMAL_ZONE", str(zone))
|
||||
assert metrics()["cpu_temp"] == 48.3
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
"""/partials/<name> dispatch and the plugin web_ui page's directory lookup."""
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
from flask import Flask
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from web_interface.blueprints import pages_v3 as module # noqa: E402
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client(tmp_path, monkeypatch):
|
||||
plugin_manager = MagicMock()
|
||||
plugin_manager.plugins_dir = tmp_path
|
||||
monkeypatch.setattr(module.pages_v3, "plugin_manager", plugin_manager, raising=False)
|
||||
monkeypatch.setattr(module.pages_v3, "config_manager",
|
||||
MagicMock(load_config=lambda: {}), raising=False)
|
||||
app = Flask(__name__, template_folder=str(
|
||||
Path(module.__file__).resolve().parents[1] / "templates"))
|
||||
app.register_blueprint(module.pages_v3)
|
||||
return app.test_client()
|
||||
|
||||
|
||||
def test_every_tab_the_old_if_chain_served_is_still_served():
|
||||
assert set(module._PARTIAL_LOADERS) == {
|
||||
"overview", "general", "display", "durations", "schedule", "plugins",
|
||||
"fonts", "logs", "raw-json", "backup-restore", "wifi", "cache",
|
||||
"operation-history", "tools",
|
||||
}
|
||||
|
||||
|
||||
def test_an_unknown_partial_is_a_404(client):
|
||||
response = client.get("/partials/nope")
|
||||
assert response.status_code == 404
|
||||
|
||||
|
||||
def test_a_failing_loader_is_one_500_that_names_the_partial(client, monkeypatch, caplog):
|
||||
def boom():
|
||||
raise RuntimeError("template exploded")
|
||||
monkeypatch.setitem(module._PARTIAL_LOADERS, "logs", boom)
|
||||
|
||||
with caplog.at_level(logging.ERROR):
|
||||
response = client.get("/partials/logs")
|
||||
|
||||
assert response.status_code == 500
|
||||
assert response.get_data(as_text=True) == "Error loading partial"
|
||||
errors = [r for r in caplog.records if r.levelno >= logging.ERROR]
|
||||
assert len(errors) == 1
|
||||
assert errors[0].getMessage() == "Error loading partial logs"
|
||||
|
||||
|
||||
def test_web_ui_page_uses_the_ledmatrix_prefix_fallback(client, tmp_path):
|
||||
web_ui = tmp_path / "ledmatrix-radar" / "web_ui"
|
||||
web_ui.mkdir(parents=True)
|
||||
(web_ui / "panel.html").write_text("<p>radar panel</p>", encoding="utf-8")
|
||||
|
||||
response = client.get("/plugin-ui/radar/web-ui/panel.html")
|
||||
|
||||
assert response.status_code == 200
|
||||
assert "radar panel" in response.get_data(as_text=True)
|
||||
@@ -51,7 +51,7 @@ def _memory(total_mb, used_mb, available_mb):
|
||||
|
||||
def _get_status(client, memory):
|
||||
# The endpoint caches for 10s; bypass so each case is measured fresh.
|
||||
with patch("web_interface.cache.get_cached", return_value=None), \
|
||||
with patch("web_interface.blueprints.api_v3.system.get_cached", return_value=None), \
|
||||
patch("psutil.virtual_memory", return_value=memory), \
|
||||
patch("psutil.cpu_percent", return_value=5.0), \
|
||||
patch("psutil.boot_time", return_value=0.0):
|
||||
@@ -90,3 +90,24 @@ def test_a_nearly_exhausted_board_reports_a_small_number(client):
|
||||
# to fork. The readout has to surface that rather than round it away.
|
||||
data = _get_status(client, _memory(total_mb=905, used_mb=800, available_mb=73))
|
||||
assert data["memory_available_mb"] == pytest.approx(73, abs=0.5)
|
||||
|
||||
|
||||
def test_status_and_live_stream_give_the_same_answer(client, monkeypatch):
|
||||
"""/system/status is built on collect_system_metrics(), so a metric has one
|
||||
value, and "could not be read" is null in both."""
|
||||
from web_interface import system_metrics
|
||||
monkeypatch.setattr(system_metrics, "_THERMAL_ZONE", "/nonexistent/thermal/temp")
|
||||
memory = _memory(total_mb=905, used_mb=620, available_mb=284)
|
||||
with patch("psutil.virtual_memory", return_value=memory), \
|
||||
patch("psutil.cpu_percent", return_value=5.0), \
|
||||
patch("psutil.boot_time", return_value=0.0), \
|
||||
patch("psutil.disk_usage", side_effect=OSError("no such mount")):
|
||||
streamed = system_metrics.collect_system_metrics()
|
||||
with patch("web_interface.blueprints.api_v3.system.get_cached", return_value=None):
|
||||
status = json.loads(client.get("/api/v3/system/status").data)["data"]
|
||||
|
||||
assert status["cpu_temp"] is None
|
||||
assert status["disk_used_percent"] is None
|
||||
for key in system_metrics.METRIC_KEYS:
|
||||
if key != "uptime_seconds": # the two reads are a moment apart
|
||||
assert status[key] == streamed[key], key
|
||||
|
||||
@@ -0,0 +1,210 @@
|
||||
"""POST /plugins/config and POST /plugins/config/reset over a real
|
||||
ConfigManager and SchemaManager.
|
||||
|
||||
The save path reshapes what the browser posts before validating it, and these
|
||||
tests pin the reshaping that validation depends on: repeats in a uniqueItems
|
||||
list are dropped, and a list the form posted as numbered fields becomes a list
|
||||
again.
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent))
|
||||
|
||||
from src.config_manager import ConfigManager # noqa: E402
|
||||
from src.plugin_system.schema_manager import SchemaManager # noqa: E402
|
||||
from test._api_v3_test_helpers import api_v3_module, build_app # noqa: E402,F401
|
||||
|
||||
PLUGIN_ID = "stocks"
|
||||
|
||||
SCHEMA = {
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled": {"type": "boolean", "default": True},
|
||||
"stock_symbols": {
|
||||
"type": "array",
|
||||
"uniqueItems": True,
|
||||
"items": {"type": "string"},
|
||||
"default": ["AAPL"],
|
||||
},
|
||||
"refresh_seconds": {"type": "integer", "default": 60},
|
||||
"api_key": {"type": "string", "x-secret": True, "default": ""},
|
||||
},
|
||||
}
|
||||
|
||||
# The shape of the news plugin's schema: a list of objects nested one level
|
||||
# down. A form posts it as feeds.custom_feeds.0.name, feeds.custom_feeds.0.url,
|
||||
# which lands as a dict keyed "0", "1", ...
|
||||
NEWS_ID = "news"
|
||||
NEWS_SCHEMA = {
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled": {"type": "boolean", "default": True},
|
||||
"feeds": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled_feeds": {
|
||||
"type": "array", "items": {"type": "string"}, "default": [],
|
||||
},
|
||||
"custom_feeds": {
|
||||
"type": "array",
|
||||
"default": [],
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"url": {"type": "string"},
|
||||
"enabled": {"type": "boolean", "default": True},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def env(tmp_path, api_v3_module):
|
||||
"""Real config and schema managers under tmp_path, on the blueprint."""
|
||||
plugins_dir = tmp_path / "plugins"
|
||||
for plugin_id, schema in ((PLUGIN_ID, SCHEMA), (NEWS_ID, NEWS_SCHEMA)):
|
||||
plugin_dir = plugins_dir / plugin_id
|
||||
plugin_dir.mkdir(parents=True)
|
||||
(plugin_dir / "config_schema.json").write_text(json.dumps(schema))
|
||||
(plugin_dir / "manifest.json").write_text(json.dumps(
|
||||
{"id": plugin_id, "name": plugin_id, "version": "1.0.0"}))
|
||||
|
||||
config_file = tmp_path / "config.json"
|
||||
config_file.write_text(json.dumps(
|
||||
{PLUGIN_ID: {"enabled": True, "stock_symbols": ["AAPL", "FNMA"]}}))
|
||||
config_manager = ConfigManager(
|
||||
config_path=str(config_file),
|
||||
secrets_path=str(tmp_path / "config_secrets.json"))
|
||||
config_manager.template_path = str(tmp_path / "no-template.json")
|
||||
|
||||
api = api_v3_module.api_v3
|
||||
api.config_manager = config_manager
|
||||
api.schema_manager = SchemaManager(plugins_dir=plugins_dir, project_root=tmp_path)
|
||||
api.plugin_manager.plugin_manifests = {PLUGIN_ID: {"id": PLUGIN_ID},
|
||||
NEWS_ID: {"id": NEWS_ID}}
|
||||
api.plugin_manager.get_plugin.return_value = None
|
||||
|
||||
class Env:
|
||||
client = build_app(api).test_client()
|
||||
plugin_manager = api.plugin_manager
|
||||
|
||||
@staticmethod
|
||||
def stored(plugin_id=PLUGIN_ID):
|
||||
return json.loads(config_file.read_text())[plugin_id]
|
||||
|
||||
Env.config_manager = config_manager
|
||||
return Env
|
||||
|
||||
|
||||
class TestUniqueItemsRepeats:
|
||||
"""A repeat in a uniqueItems list is dropped, not a failed save."""
|
||||
|
||||
def test_form_post_repeating_a_saved_symbol_saves(self, env):
|
||||
response = env.client.post(
|
||||
f"/api/v3/plugins/config?plugin_id={PLUGIN_ID}",
|
||||
data={"stock_symbols": "AAPL, FNMA, FNMA"})
|
||||
|
||||
assert response.status_code == 200, response.get_json()
|
||||
assert env.stored()["stock_symbols"] == ["AAPL", "FNMA"]
|
||||
|
||||
def test_json_post_keeps_first_occurrence_order(self, env):
|
||||
response = env.client.post("/api/v3/plugins/config", json={
|
||||
"plugin_id": PLUGIN_ID,
|
||||
"config": {"stock_symbols": ["TSLA", "AAPL", "TSLA", "FNMA"]},
|
||||
})
|
||||
|
||||
assert response.status_code == 200, response.get_json()
|
||||
assert env.stored()["stock_symbols"] == ["TSLA", "AAPL", "FNMA"]
|
||||
|
||||
|
||||
class TestNumberedFieldsBecomeLists:
|
||||
"""The news plugin's custom feeds, posted one field per row."""
|
||||
|
||||
FEEDS = [{"name": "Local", "url": "https://example.com/local.xml", "enabled": True},
|
||||
{"name": "Tech", "url": "https://example.com/tech.xml", "enabled": False}]
|
||||
|
||||
def test_form_post(self, env):
|
||||
response = env.client.post(f"/api/v3/plugins/config?plugin_id={NEWS_ID}", data={
|
||||
"feeds.custom_feeds.0.name": "Local",
|
||||
"feeds.custom_feeds.0.url": "https://example.com/local.xml",
|
||||
"feeds.custom_feeds.0.enabled": "on",
|
||||
"feeds.custom_feeds.1.name": "Tech",
|
||||
"feeds.custom_feeds.1.url": "https://example.com/tech.xml",
|
||||
})
|
||||
|
||||
assert response.status_code == 200, response.get_json()
|
||||
assert env.stored(NEWS_ID)["feeds"]["custom_feeds"] == self.FEEDS
|
||||
|
||||
def test_json_post(self, env):
|
||||
response = env.client.post("/api/v3/plugins/config", json={
|
||||
"plugin_id": NEWS_ID,
|
||||
"config": {"feeds": {"custom_feeds": {"1": self.FEEDS[1], "0": self.FEEDS[0]}}},
|
||||
})
|
||||
|
||||
assert response.status_code == 200, response.get_json()
|
||||
assert env.stored(NEWS_ID)["feeds"]["custom_feeds"] == self.FEEDS
|
||||
|
||||
|
||||
class TestReset:
|
||||
"""POST /plugins/config/reset saves the way every other plugin save does."""
|
||||
|
||||
def test_reset_saves_atomically_with_a_backup(self, env, monkeypatch):
|
||||
cm = env.config_manager
|
||||
calls = []
|
||||
real_atomic = cm.save_config_atomic
|
||||
|
||||
def spy(config, create_backup=True, **kwargs):
|
||||
calls.append(create_backup)
|
||||
return real_atomic(config, create_backup=create_backup, **kwargs)
|
||||
|
||||
def no_plain_save(_config):
|
||||
raise AssertionError("reset bypassed the atomic save")
|
||||
|
||||
monkeypatch.setattr(cm, "save_config_atomic", spy)
|
||||
monkeypatch.setattr(cm, "save_config", no_plain_save)
|
||||
|
||||
response = env.client.post("/api/v3/plugins/config/reset",
|
||||
json={"plugin_id": PLUGIN_ID})
|
||||
|
||||
assert response.status_code == 200, response.get_json()
|
||||
assert calls == [True]
|
||||
assert env.stored()["stock_symbols"] == ["AAPL"]
|
||||
|
||||
def test_reset_notifies_the_plugin_with_its_prepared_config(self, env):
|
||||
plugin = MagicMock()
|
||||
env.plugin_manager.get_plugin.return_value = plugin
|
||||
env.plugin_manager.prepare_plugin_config.side_effect = (
|
||||
lambda _pid, raw: {**raw, "prepared": True})
|
||||
|
||||
response = env.client.post("/api/v3/plugins/config/reset",
|
||||
json={"plugin_id": PLUGIN_ID})
|
||||
|
||||
assert response.status_code == 200, response.get_json()
|
||||
handed_over = plugin.on_config_change.call_args.args[0]
|
||||
assert handed_over["prepared"] is True
|
||||
assert handed_over["stock_symbols"] == ["AAPL"]
|
||||
|
||||
def test_a_failed_save_is_reported(self, env, monkeypatch):
|
||||
failed = MagicMock(message="disk full")
|
||||
failed.status.value = "failed"
|
||||
monkeypatch.setattr(env.config_manager, "save_config_atomic",
|
||||
MagicMock(return_value=failed))
|
||||
|
||||
response = env.client.post("/api/v3/plugins/config/reset",
|
||||
json={"plugin_id": PLUGIN_ID})
|
||||
|
||||
assert response.status_code == 500
|
||||
assert "disk full" in response.get_json()["message"]
|
||||
@@ -0,0 +1,58 @@
|
||||
"""coerce_array_shapes: position-keyed dicts back into lists before validation."""
|
||||
|
||||
from src.web_interface.config_arrays import coerce_array_shapes
|
||||
|
||||
COLOR = {"type": "array", "items": {"type": "integer"},
|
||||
"minItems": 3, "maxItems": 3, "default": [255, 255, 255]}
|
||||
|
||||
|
||||
def test_position_keys_become_a_list_in_numeric_order():
|
||||
config = {"tags": {"10": "k", "2": "c", "0": "a"}}
|
||||
coerce_array_shapes(config, {"tags": {"type": "array"}})
|
||||
assert config["tags"] == ["a", "c", "k"]
|
||||
|
||||
|
||||
def test_an_empty_dict_becomes_an_empty_list():
|
||||
config = {"tags": {}}
|
||||
coerce_array_shapes(config, {"tags": {"type": "array"}})
|
||||
assert config["tags"] == []
|
||||
|
||||
|
||||
def test_a_dict_with_other_keys_is_left_for_validation_to_report():
|
||||
config = {"tags": {"0": "a", "name": "b"}}
|
||||
coerce_array_shapes(config, {"tags": {"type": "array"}})
|
||||
assert config["tags"] == {"0": "a", "name": "b"}
|
||||
|
||||
|
||||
def test_element_types_are_left_to_normalization():
|
||||
config = {"color": {"0": "1", "1": "2", "2": "3"}}
|
||||
coerce_array_shapes(config, {"color": COLOR})
|
||||
assert config["color"] == ["1", "2", "3"]
|
||||
|
||||
|
||||
def test_nested_objects_and_array_items_are_walked():
|
||||
schema = {"feeds": {"type": "object", "properties": {
|
||||
"custom_feeds": {"type": "array", "items": {"type": "object", "properties": {
|
||||
"tags": {"type": "array"},
|
||||
}}},
|
||||
}}}
|
||||
config = {"feeds": {"custom_feeds": {"0": {"tags": {"0": "news"}}}}}
|
||||
coerce_array_shapes(config, schema)
|
||||
assert config == {"feeds": {"custom_feeds": [{"tags": ["news"]}]}}
|
||||
|
||||
|
||||
def test_a_short_form_list_takes_the_default_only_when_asked():
|
||||
config = {"color": ["10", "20"]}
|
||||
coerce_array_shapes(config, {"color": COLOR})
|
||||
assert config["color"] == ["10", "20"]
|
||||
|
||||
coerce_array_shapes(config, {"color": COLOR}, short_lists_take_default=True)
|
||||
assert config["color"] == [255, 255, 255]
|
||||
assert config["color"] is not COLOR["default"]
|
||||
|
||||
|
||||
def test_a_default_too_short_itself_is_not_used():
|
||||
schema = {"color": dict(COLOR, default=[0])}
|
||||
config = {"color": ["10"]}
|
||||
coerce_array_shapes(config, schema, short_lists_take_default=True)
|
||||
assert config["color"] == ["10"]
|
||||
@@ -1,11 +1,9 @@
|
||||
"""Tests for dedup_unique_arrays used by save_plugin_config.
|
||||
"""Tests for dedup_unique_arrays, which the plugin-config save path
|
||||
(_prepare_plugin_config_for_save in api_v3/plugins.py) runs before validation.
|
||||
|
||||
Validates that arrays with uniqueItems: true in the JSON schema have
|
||||
duplicates removed before validation, preventing spurious validation
|
||||
failures when form merging introduces duplicate entries.
|
||||
|
||||
Tests import the production function from src.web_interface.validators
|
||||
to ensure they exercise the real code path.
|
||||
Arrays with uniqueItems: true in the JSON schema lose their duplicates, so a
|
||||
repeat introduced by form merging does not fail validation.
|
||||
test_api_v3_plugin_config_save.py checks the same through the endpoint.
|
||||
"""
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user