mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 06:15:09 +00:00
* fix(config): load_config hands each caller a private copy ConfigManager.load_config() returned its cached self.config itself (the mtime fast path from #410 kept the full path's aliasing). Web handlers edit what they load and then validate: the plugin form save applies the posted fields to the loaded section (a shallow .copy(), so nested dicts were the cache's own), and save_main_config sets its checkboxes before it checks auto_update_channel. When the save was refused, the edit stayed in the cache the fast path serves, and the next save of any other setting wrote it to config.json: the refused value, and a nested secret typed into the same form (mqtt.password, league.espn_s2, flightaware.api_key) in plain text, since it never reached config_secrets.json to be stripped. The form also reloaded showing the refused values. load_config() now returns a private copy on both paths, and save_config/save_config_atomic keep a copy of what they were given, so nothing a caller edits reaches the cache unless it is saved. Fixing it here rather than in each handler covers every route that edits before it validates. No caller relies on editing the cache without saving: every src/ and web_interface/ caller either reads, or saves the dict it edited. get_config() still returns the live dict for the display process's readers. The copy is a pickle round trip: on a Pi 4 with its real 64 KiB config, 2.0 ms against 6.9 ms for copy.deepcopy (json round trip 3.4 ms). Two tests asserted the aliasing itself and now assert a copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): GET /plugins/config masks secrets and refuses core sections The route returned the plugin's section as load_config() has it, with config_secrets.json merged in: API keys and tokens went out in plain text. #276 masked them here; #330's rewrite of the route dropped it, while the settings page and GET /config/secrets kept masking. It also took any plugin_id, so ?plugin_id=web_auth returned the login's cookie-signing key and password hash, and ?plugin_id=github the Plugin Store token, which GET /config/main strips and redacts. The route now refuses what _non_plugin_id_error refuses for reset and uninstall (core sections, malformed ids) with a 400, and blanks x-secret fields with mask_secret_fields after the defaults merge, as the page does. A plugin with no schema has its credential-named fields blanked by _redact_credentials, as GET /config/main does. Blank rather than the bullets of GET /config/secrets: the save drops a blank secret as "unchanged" (remove_empty_secrets) but would store the bullets, so the response must post back as it came. Tested: GET, then POST the response unchanged, keeps every stored secret. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): parse a table row's cells against the list's item schema An array of objects drawn as a table posts each cell as "cities.0.timezone". _get_schema_property stopped at "cities" (an array, not an object with properties), so _parse_form_value_with_schema got no schema for the cell and guessed: a blank optional text cell became None and a text cell holding digits became an int. Validation refused both, so every save of the page failed for as long as such a row existed -- geochron's city without a timezone, a countdown named "2027". A secret cell is always drawn blank, so a plugin with secrets in its rows could not be saved from the form at all. The lookup now steps from an index segment into the array's items: to the item schema itself for "color.2", into its properties for a row cell. Number, boolean and required cells convert as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): a blank secret field saves as "unchanged", required or not The settings page draws a stored secret blank (mask_secret_fields) and posts the blank back. _parse_form_value_with_schema turned a blank optional string into "" -- which the save drops as unchanged (remove_empty_secrets) -- but a blank required one into None. For a secret that is required with no default (youtube-stats' api_key) that None failed validation, so every save of the page was refused until the key was typed in again. A blank text secret (x-secret, type string) now parses to "", whatever its required list says; a list or object secret keeps getting [] or {}, which the save drops the same way. Not _SKIP_FIELD: skipping keeps the value load_config() merged in, and the save would then write it back to config_secrets.json -- after a secret change the cached section can still hold the old one, so that write reverted it. A test covers that sequence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): POST /plugins/config refuses core sections and malformed ids Reset and uninstall check the plugin id with _non_plugin_id_error; the save did not. {"plugin_id": "display", "config": {...}} found no schema, so nothing was validated or filtered, and the body was merged into the core display section along with "enabled": true -- rows: "banana" included. A plugin_id that was not a string (a list, an object, a number) reached config.get() or the schema lookup, raised TypeError, and came back as a 500. Both the JSON and the form path now call _non_plugin_id_error first and answer its 400. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): a text field keeps "true", "[1, 2]" and "{}" as typed _parse_form_value_with_schema guessed before it consulted the schema: "true"/"false" became booleans, and a value starting with "[" or "{" that parsed as JSON became a list or object, whatever the field's type. A text setting holding "true", "False", "[1, 2]" or "{}" was then refused by validation ("Expected type string, got bool"), and the save with it. A field whose schema type is string, or string-or-null, now returns the posted text as it came. Every other type goes through the conversions as before; numbers in text fields were already left alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(config): copy the cached config without pickle _private_copy was a pickle round trip. It only ever unpickled bytes it had just made from our own dict, so nothing untrusted reached it, but it put pickle in the config path and Codacy failed the PR for it (B301/B403). The config is JSON data, so copying its dicts and lists is a full copy; every other value is immutable. Measured on ledpi (Pi 4) with its real 60 KiB config: 2.11 ms, against 1.92 ms for pickle and 6.75 ms for copy.deepcopy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): describe the config copy without pickle Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
167 lines
7.0 KiB
Python
167 lines
7.0 KiB
Python
"""Tests for load_config's mtime fast path (src/config_manager.py).
|
|
|
|
load_config used to re-read + re-parse config.json, the template (with a
|
|
recursive migration diff) and secrets on EVERY call — ~30 web request
|
|
handlers call it, some 2-3x per request. The fast path skips all of it
|
|
when the three files' (mtime_ns, size) signatures are unchanged.
|
|
|
|
The invariant that matters most: cross-process freshness — a save from
|
|
the web process must be picked up by the display process's next load.
|
|
That's guaranteed because the signature is re-stat'd on every call.
|
|
"""
|
|
|
|
import json
|
|
import os
|
|
import sys
|
|
|
|
import pytest
|
|
|
|
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
|
|
|
from src.config_manager import ConfigManager # noqa: E402
|
|
|
|
|
|
@pytest.fixture
|
|
def mgr(tmp_path):
|
|
config = tmp_path / "config.json"
|
|
secrets = tmp_path / "secrets.json"
|
|
template = tmp_path / "template.json"
|
|
config.write_text(json.dumps({"display": {"brightness": 90}, "timezone": "UTC"}))
|
|
secrets.write_text(json.dumps({"weather": {"api_key": "sek"}}))
|
|
template.write_text(json.dumps({"display": {"brightness": 90}, "timezone": "UTC"}))
|
|
m = ConfigManager(config_path=str(config), secrets_path=str(secrets))
|
|
m.template_path = str(template)
|
|
return m, config, secrets, template
|
|
|
|
|
|
def _count_opens(monkeypatch, mgr_paths):
|
|
"""Count open() calls hitting the config files."""
|
|
counts = {"n": 0}
|
|
real_open = open
|
|
|
|
def counting_open(file, *args, **kwargs):
|
|
if str(file) in mgr_paths:
|
|
counts["n"] += 1
|
|
return real_open(file, *args, **kwargs)
|
|
|
|
import builtins
|
|
monkeypatch.setattr(builtins, "open", counting_open)
|
|
return counts
|
|
|
|
|
|
class TestFastPath:
|
|
def test_unchanged_files_are_not_reread(self, mgr, monkeypatch):
|
|
m, config, secrets, template = mgr
|
|
first = m.load_config()
|
|
assert first["weather"]["api_key"] == "sek" # secrets merged
|
|
counts = _count_opens(monkeypatch, {str(config), str(secrets), str(template)})
|
|
for _ in range(10):
|
|
again = m.load_config()
|
|
assert counts["n"] == 0, "fast path must not re-open any config file"
|
|
assert again == first
|
|
assert again is not first # each caller gets its own copy, see below
|
|
|
|
def test_config_change_triggers_reload(self, mgr):
|
|
m, config, secrets, template = mgr
|
|
m.load_config()
|
|
data = json.loads(config.read_text())
|
|
data["display"]["brightness"] = 55
|
|
config.write_text(json.dumps(data))
|
|
os.utime(config, (os.stat(config).st_atime, os.stat(config).st_mtime + 2))
|
|
assert m.load_config()["display"]["brightness"] == 55
|
|
|
|
def test_secrets_change_triggers_reload(self, mgr):
|
|
m, config, secrets, template = mgr
|
|
m.load_config()
|
|
secrets.write_text(json.dumps({"weather": {"api_key": "NEW"}}))
|
|
os.utime(secrets, (os.stat(secrets).st_atime, os.stat(secrets).st_mtime + 2))
|
|
assert m.load_config()["weather"]["api_key"] == "NEW"
|
|
|
|
def test_template_change_triggers_reload_and_migration(self, mgr):
|
|
m, config, secrets, template = mgr
|
|
m.load_config()
|
|
template.write_text(json.dumps({
|
|
"display": {"brightness": 90}, "timezone": "UTC",
|
|
"brand_new_key": {"added": True}}))
|
|
os.utime(template, (os.stat(template).st_atime, os.stat(template).st_mtime + 2))
|
|
reloaded = m.load_config()
|
|
assert reloaded.get("brand_new_key") == {"added": True}
|
|
|
|
def test_same_second_edit_detected_via_mtime_ns_or_size(self, mgr):
|
|
"""Coarse-mtime same-second edits: size difference still busts it."""
|
|
m, config, secrets, template = mgr
|
|
m.load_config()
|
|
st = os.stat(config)
|
|
data = json.loads(config.read_text())
|
|
data["timezone"] = "America/New_York" # different byte length
|
|
config.write_text(json.dumps(data))
|
|
os.utime(config, (st.st_atime, st.st_mtime)) # force same mtime
|
|
assert m.load_config()["timezone"] == "America/New_York"
|
|
|
|
|
|
class TestCallersGetACopy:
|
|
"""A web handler edits what load_config returned, then validates. When
|
|
validation failed, the edit stayed in the cache the fast path serves, and
|
|
the next unrelated save wrote it -- a nested secret included, in plain
|
|
text, because it had never reached config_secrets.json to be stripped."""
|
|
|
|
def test_editing_a_loaded_config_does_not_change_the_next_load(self, mgr):
|
|
m, config, secrets, template = mgr
|
|
loaded = m.load_config()
|
|
loaded["display"]["brightness"] = 1
|
|
loaded["weather"]["api_key"] = "typed-but-never-saved"
|
|
again = m.load_config()
|
|
assert again["display"]["brightness"] == 90
|
|
assert again["weather"]["api_key"] == "sek"
|
|
|
|
def test_the_full_path_also_returns_a_copy(self, mgr):
|
|
m, config, secrets, template = mgr
|
|
m.load_config()["display"]["brightness"] = 1 # first load: full path
|
|
assert m.load_config()["display"]["brightness"] == 90
|
|
|
|
def test_an_edit_never_reaches_a_later_save(self, mgr):
|
|
m, config, secrets, template = mgr
|
|
m.load_config()["display"]["new_secret"] = "hunter2" # then bailed out
|
|
m.save_config(m.load_config()) # some other handler saves
|
|
assert "hunter2" not in config.read_text()
|
|
|
|
|
|
class TestSaveCoherence:
|
|
def test_save_config_then_load_returns_saved_data(self, mgr, monkeypatch):
|
|
m, config, secrets, template = mgr
|
|
m.load_config()
|
|
new = {"display": {"brightness": 42}, "timezone": "UTC",
|
|
"weather": {"api_key": "sek"}}
|
|
m.save_config(new)
|
|
counts = _count_opens(monkeypatch, {str(config), str(secrets), str(template)})
|
|
loaded = m.load_config()
|
|
assert loaded["display"]["brightness"] == 42
|
|
assert loaded["weather"]["api_key"] == "sek" # secrets survive in memory
|
|
assert counts["n"] == 0 # signature refreshed by save; no re-read
|
|
|
|
def test_the_saved_dict_does_not_become_the_cache(self, mgr):
|
|
m, config, secrets, template = mgr
|
|
m.load_config()
|
|
new = {"display": {"brightness": 42}, "timezone": "UTC",
|
|
"weather": {"api_key": "sek"}}
|
|
m.save_config(new)
|
|
new["display"]["brightness"] = 7 # the caller keeps using its dict
|
|
assert m.load_config()["display"]["brightness"] == 42
|
|
|
|
def test_cross_process_save_is_picked_up(self, mgr):
|
|
"""Another process writing config.json (different mtime) must bust
|
|
this process's fast path — the core cross-process guarantee."""
|
|
m, config, secrets, template = mgr
|
|
m.load_config()
|
|
other = ConfigManager(config_path=str(config), secrets_path=str(secrets))
|
|
other.template_path = str(template)
|
|
other.load_config()
|
|
other.save_config({"display": {"brightness": 11}, "timezone": "UTC",
|
|
"weather": {"api_key": "sek"}})
|
|
os.utime(config, (os.stat(config).st_atime, os.stat(config).st_mtime + 2))
|
|
assert m.load_config()["display"]["brightness"] == 11
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(pytest.main([__file__, "-v"]))
|