mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-08-07 19:58:08 +00:00
* ci: run the whole test tree and make the plugin-safety job assert something real The unit-tests CI job ran an explicit 24-file allowlist that had rotted: 63 of 90 test files (display, vegas, store manager, web API, web_interface) never ran on a PR. The job now runs all of test/ (minus test/plugins, which the plugin-safety job owns) so new test files are enrolled by default and any exclusion needs a visible, commented --ignore. The plugin-safety job was a green no-op: plugins/ is empty in CI, so every test skipped with 'Manifest not found'. It now renders a bundled deterministic fixture plugin (test/fixtures/plugins/ci-fixture-plugin, golden images included for all 8 default sizes) via LEDMATRIX_PLUGINS_DIR, and sets LEDMATRIX_REQUIRE_PLUGINS=1 so discovering zero plugins fails loudly instead of skipping green. The per-plugin suites document that they target dev machines with real plugins installed. Coverage is now measured and enforced in exactly one place — the CI unit-tests step (--cov=src --cov=web_interface --cov-fail-under=45, from a measured 47% baseline). pytest.ini previously declared --cov-fail-under=30 but CI always passed --no-cov, so the gate had never run anywhere; local pytest is now coverage-free and fast. Enabling the 63 unenrolled files surfaced three cases of test rot, fixed here: test_display_controller_vegas_tick.py could not collect without the hardware rgbmatrix module (now uses the emulator convention), the state-reconciliation unrecoverable-cache tests broke when production added the is_plugin_uninstalled tombstone check (bare Mock returned truthy), and test_get_system_status assumed the optional psutil dependency (now installed via requirements-test.txt and guarded by importorskip). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh * test: replace can't-fail tests with real assertions test_font_manager.py was 5 of 6 tests shaped as 'try: call(); assert True / except: assert True' — running in CI while unable to fail on any regression. Rewritten against the real FontManager API and the bundled assets/fonts: returned font types, cache-hit identity, distinct entries per size, default-font fallback for unknown families and corrupt files (recorded in failed_loads), BDF native-size reading, text measurement, and cache lifecycle. test_display_manager.py's test_draw_text ended in 'assert True'; it now renders onto a known-black canvas and asserts pixels were actually lit — which required un-breaking the fixture's freetype MagicMock so draw_text's isinstance check doesn't silently swallow the draw. test_display_controller.py carried a permanently-skipped test whose skip reason already declared it redundant; deleted. Both display test files now set EMULATOR=true before importing display_manager (the same convention as test_display_dirty_tracking.py) so they collect standalone instead of depending on which test module imports display_manager first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh * test: cover the untested fragile logic (compatibility gate, secrets, config merges, durations, skin cards) New unit tests for pure or filesystem-only logic that previously had zero direct coverage: - test_compatibility.py: the semver install gate (parse_semver suffix handling, every range operator, TRUSTWORTHY_FLOOR behavior for cores reporting untrustworthy versions, 'more restrictive wins', and the malformed-manifest shapes that used to raise). - test/web_interface/test_secret_helpers.py: the canonical x-secret helpers — find/separate/mask/remove, array-item secrets, no input mutation, and a separate->recombine round-trip. - test/web_interface/test_api_v3_helpers.py: the module-level helpers behind the plugin config save endpoint (_is_plugin_update_available, _coerce_to_bool including the int==1 quirk, deep_merge including its shared-subtree shallowness, _parse_form_value, dotted-key-aware _get_schema_property/_set_nested_value). - test_base_plugin_duration.py: get_display_duration's full coercion ladder (instance attr -> config -> 15.0), including the bool-is-int quirk where display_duration=True means one second. - test_config_manager_secrets.py: the secrets round-trip — deep-merge on load, strip on save, group pruning, the load fast path — and two characterized sharp edges marked SUSPECTED BUG: an unreadable secrets file at save time writes secrets into config.json in plaintext, and a same-mtime-same-size content swap is served stale. - test_schema_manager_merge.py: merge_with_defaults branch behavior (None replacement vs falsey preservation, dict-vs-scalar mismatches, arrays replaced wholesale, defaults never mutated). - test_skin_system.py (extended): render_skin_card shares _render_game's 3-strike counter but never resets it on success — the asymmetry is pinned in both directions, along with card fallthrough and the disable interaction between the two paths. Suspected bugs are characterized, not fixed — each carries a comment so a future behavior change is deliberate rather than accidental. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh * test: add drift guards for cross-file contracts Three guard suites that pin contracts spanning multiple files, where one side changing unilaterally breaks the other silently: - test_version_comparison_consistency.py: the repo's four version comparators (compatibility.parse_semver, api_v3's packaging-based _is_plugin_update_available, store_manager update_plugin's raw string equality, skin_runtime._major) answer differently on the same inputs. A table pins each one's verdict; update_plugin is driven through its real code path to show the SUSPECTED BUGs: 'v1.2.0' vs '1.2.0' triggers a full reinstall the UI calls unnecessary, and a locally-ahead plugin gets downgraded. A pairwise-ordering check keeps parse_semver agreeing with packaging on plain X.Y.Z. - test/web_interface/test_secret_separation_parity.py: api_v3.py carries three inline copies of find_secret_fields/separate_secrets that lack the canonical module's array-item support. The copy count is asserted exact (it may only go down; new copies must import src/web_interface/secret_helpers), the missing-array-support gap is asserted so it can't grow silently, and the canonical behavior that migration will adopt is documented executably. - test_discovery_path_contract.py: the three 'where is plugin X' resolvers (PluginManager discovery, StoreManager._find_plugin_path, SchemaManager.get_schema_path) agree on the configured directory, and their divergent fallback chains are characterized. Also pins the .standalone-backup- naming contract shared by store rollback and discovery, and _resolve_skin_target's path-traversal rejection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh * test: address review feedback — fixture lifecycle, test names, ClassVar - ci-fixture-plugin: call display_manager.clear() before rendering (per plugin guidelines — the fixture should model a well-behaved plugin), add a class docstring, and document why Pillow is deliberately not pinned in its requirements.txt (core dependency; harness installs nothing). - Rename two tests whose names contradicted their assertions: test_unparseable_core_version_is_compatible -> test_unparseable_core_with_high_floor_is_blocked, and test_unreadable_secrets_file... -> test_corrupt_secrets_file... - Annotate TestGetSchemaProperty.SCHEMA as ClassVar (RUF012). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh * ci: allow manual test.yml runs via workflow_dispatch Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh * fix: unify version comparison, refuse secret-leaking saves, reset skin strikes on card success Fixes the three suspected bugs this PR's characterization tests pinned, flipping those tests to assert the corrected behavior: - plugins/store: ONE shared update comparator. New compatibility.is_update_available() (PEP 440 via packaging) is now used by both the web UI's update badge (api_v3._is_plugin_update_available is a thin alias) and store_manager.update_plugin's reinstall decision. Previously update_plugin used raw string equality: 'v1.2.0' vs '1.2.0' triggered a full reinstall the UI called unnecessary, and a locally- ahead plugin (2.0.0 installed, registry 1.9.0) was silently DOWNGRADED. Now equivalent spellings skip the reinstall and locally-ahead versions are never downgraded; unparseable versions still reconcile by reinstalling from the registry. - config: save_config and save_config_atomic now refuse (ConfigError) when config_secrets.json exists but cannot be loaded. Both previously proceeded without stripping, writing the merged secrets into config.json in plaintext. The shared _load_secrets_for_save() helper raises with an actionable message instead; a missing secrets file is still fine (nothing to strip), and _migrate_config's catch-all keeps boot resilient. - skins: render_skin_card resets _skin_failures on both success paths (vegas card returned, or mode renderer handled), mirroring _render_game. Transient card failures no longer accumulate across a session until they permanently disable a working skin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh * fix: harden shared comparator edges from review - is_update_available: reject truthy non-string versions (a malformed manifest can carry a number; packaging raises TypeError on those) by surfacing the mismatch instead of raising. - store_manager.update_plugin: drop the truthiness gate around the comparator so a missing version on either side follows the shared 'no update' verdict, keeping the store consistent with the UI badge; a missing manifest still uses the reinstall recovery path. - config_manager._load_secrets_for_save: catch only expected read/parse failures (OSError/ValueError/RecursionError) so implementation bugs propagate as themselves, and log with traceback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh --------- Co-authored-by: Claude <noreply@anthropic.com>
242 lines
9.4 KiB
Python
242 lines
9.4 KiB
Python
"""
|
|
Tests for src/web_interface/secret_helpers.py — the canonical secret
|
|
identification / separation / masking helpers.
|
|
|
|
This module is the extracted single source of truth for x-secret handling,
|
|
but until now had zero test coverage (only ``mask_secret_fields`` is even
|
|
imported by production code, from pages_v3). api_v3.py still carries three
|
|
inline re-implementations of ``find_secret_fields``/``separate_secrets`` —
|
|
see test_secret_separation_parity.py — so pinning the canonical behavior
|
|
here is a precondition for ever migrating those copies.
|
|
"""
|
|
|
|
import copy
|
|
|
|
from src.web_interface.secret_helpers import (
|
|
find_secret_fields,
|
|
separate_secrets,
|
|
mask_secret_fields,
|
|
mask_all_secret_values,
|
|
remove_empty_secrets,
|
|
)
|
|
|
|
|
|
SCHEMA_PROPS = {
|
|
"api_key": {"type": "string", "x-secret": True},
|
|
"city": {"type": "string"},
|
|
"auth": {
|
|
"type": "object",
|
|
"properties": {
|
|
"token": {"type": "string", "x-secret": True},
|
|
"username": {"type": "string"},
|
|
},
|
|
},
|
|
"accounts": {
|
|
"type": "array",
|
|
"items": {
|
|
"type": "object",
|
|
"properties": {
|
|
"name": {"type": "string"},
|
|
"token": {"type": "string", "x-secret": True},
|
|
},
|
|
},
|
|
},
|
|
"recovery_codes": {
|
|
"type": "array",
|
|
"items": {"type": "string", "x-secret": True},
|
|
},
|
|
}
|
|
|
|
|
|
class TestFindSecretFields:
|
|
def test_top_level_secret(self):
|
|
assert "api_key" in find_secret_fields(SCHEMA_PROPS)
|
|
|
|
def test_non_secret_not_included(self):
|
|
assert "city" not in find_secret_fields(SCHEMA_PROPS)
|
|
|
|
def test_nested_object_secret_uses_dot_path(self):
|
|
assert "auth.token" in find_secret_fields(SCHEMA_PROPS)
|
|
assert "auth.username" not in find_secret_fields(SCHEMA_PROPS)
|
|
|
|
def test_array_item_object_secret_uses_bracket_path(self):
|
|
assert "accounts[].token" in find_secret_fields(SCHEMA_PROPS)
|
|
|
|
def test_array_of_secrets_uses_bracket_path(self):
|
|
assert "recovery_codes[]" in find_secret_fields(SCHEMA_PROPS)
|
|
|
|
def test_full_set(self):
|
|
assert find_secret_fields(SCHEMA_PROPS) == {
|
|
"api_key", "auth.token", "accounts[].token", "recovery_codes[]",
|
|
}
|
|
|
|
def test_non_dict_properties_tolerated(self):
|
|
assert find_secret_fields({"weird": "not-a-dict"}) == set()
|
|
|
|
def test_non_dict_input_returns_empty(self):
|
|
assert find_secret_fields(None) == set()
|
|
assert find_secret_fields([]) == set()
|
|
|
|
|
|
class TestSeparateSecrets:
|
|
def test_flat_partition(self):
|
|
regular, secrets = separate_secrets(
|
|
{"api_key": "s3cret", "city": "Austin"}, {"api_key"})
|
|
assert regular == {"city": "Austin"}
|
|
assert secrets == {"api_key": "s3cret"}
|
|
|
|
def test_nested_partition(self):
|
|
config = {"auth": {"token": "t0k", "username": "chuck"}}
|
|
regular, secrets = separate_secrets(config, {"auth.token"})
|
|
assert regular == {"auth": {"username": "chuck"}}
|
|
assert secrets == {"auth": {"token": "t0k"}}
|
|
|
|
def test_empty_nested_dicts_pruned_from_regular(self):
|
|
# A dict that is all secrets leaves nothing behind on the regular
|
|
# side — the key must be dropped, not kept as {}.
|
|
config = {"auth": {"token": "t0k"}}
|
|
regular, secrets = separate_secrets(config, {"auth.token"})
|
|
assert regular == {}
|
|
assert secrets == {"auth": {"token": "t0k"}}
|
|
|
|
def test_whole_array_secret(self):
|
|
config = {"recovery_codes": ["a", "b"], "city": "Austin"}
|
|
regular, secrets = separate_secrets(config, {"recovery_codes[]"})
|
|
assert regular == {"city": "Austin"}
|
|
assert secrets == {"recovery_codes": ["a", "b"]}
|
|
|
|
def test_array_item_secrets_produce_parallel_lists(self):
|
|
# Per-item secrets keep the arrays index-aligned so they can be
|
|
# recombined: regular gets the stripped items, secrets a parallel
|
|
# list of the extracted values.
|
|
config = {"accounts": [
|
|
{"name": "a", "token": "ta"},
|
|
{"name": "b", "token": "tb"},
|
|
]}
|
|
regular, secrets = separate_secrets(config, {"accounts[].token"})
|
|
assert regular == {"accounts": [{"name": "a"}, {"name": "b"}]}
|
|
assert secrets == {"accounts": [{"token": "ta"}, {"token": "tb"}]}
|
|
|
|
def test_array_item_non_dict_items_get_placeholder(self):
|
|
config = {"accounts": [{"name": "a", "token": "ta"}, "oddball"]}
|
|
regular, secrets = separate_secrets(config, {"accounts[].token"})
|
|
assert regular == {"accounts": [{"name": "a"}, "oddball"]}
|
|
assert secrets == {"accounts": [{"token": "ta"}, {}]}
|
|
|
|
def test_array_without_secret_paths_stays_regular(self):
|
|
config = {"teams": ["DAL", "HOU"]}
|
|
regular, secrets = separate_secrets(config, {"api_key"})
|
|
assert regular == {"teams": ["DAL", "HOU"]}
|
|
assert secrets == {}
|
|
|
|
def test_round_trip_loses_nothing(self):
|
|
# separate + naive recombine must reconstruct the original config.
|
|
config = {
|
|
"api_key": "k",
|
|
"city": "Austin",
|
|
"auth": {"token": "t", "username": "chuck"},
|
|
"recovery_codes": ["a", "b"],
|
|
}
|
|
paths = find_secret_fields(SCHEMA_PROPS)
|
|
regular, secrets = separate_secrets(copy.deepcopy(config), paths)
|
|
|
|
def recombine(reg, sec):
|
|
out = copy.deepcopy(reg)
|
|
for k, v in sec.items():
|
|
if isinstance(v, dict) and isinstance(out.get(k), dict):
|
|
out[k] = recombine(out[k], v)
|
|
else:
|
|
out[k] = v
|
|
return out
|
|
|
|
assert recombine(regular, secrets) == config
|
|
|
|
|
|
class TestMaskSecretFields:
|
|
def test_masks_present_secret_to_empty_string(self):
|
|
result = mask_secret_fields({"api_key": "s3cret"}, SCHEMA_PROPS)
|
|
assert result["api_key"] == ""
|
|
|
|
def test_leaves_non_secret_untouched(self):
|
|
result = mask_secret_fields({"city": "Austin"}, SCHEMA_PROPS)
|
|
assert result["city"] == "Austin"
|
|
|
|
def test_none_and_empty_left_alone(self):
|
|
result = mask_secret_fields({"api_key": None}, SCHEMA_PROPS)
|
|
assert result["api_key"] is None
|
|
result = mask_secret_fields({"api_key": ""}, SCHEMA_PROPS)
|
|
assert result["api_key"] == ""
|
|
|
|
def test_falsey_but_set_values_are_masked(self):
|
|
# 0 and False are real values; the check is `is not None and != ''`.
|
|
# Note False == '' is False in Python, so False IS masked; 0 == '' is
|
|
# also False, so 0 is masked too.
|
|
result = mask_secret_fields({"api_key": 0}, SCHEMA_PROPS)
|
|
assert result["api_key"] == ""
|
|
result = mask_secret_fields({"api_key": False}, SCHEMA_PROPS)
|
|
assert result["api_key"] == ""
|
|
|
|
def test_nested_object_masked_without_mutating_input(self):
|
|
config = {"auth": {"token": "t0k", "username": "chuck"}}
|
|
original = copy.deepcopy(config)
|
|
result = mask_secret_fields(config, SCHEMA_PROPS)
|
|
assert result["auth"]["token"] == ""
|
|
assert result["auth"]["username"] == "chuck"
|
|
assert config == original # input not mutated
|
|
|
|
def test_array_of_secrets_masked_elementwise(self):
|
|
result = mask_secret_fields(
|
|
{"recovery_codes": ["a", "b"]}, SCHEMA_PROPS)
|
|
assert result["recovery_codes"] == ["", ""]
|
|
|
|
def test_array_of_objects_masked_per_item(self):
|
|
config = {"accounts": [{"name": "a", "token": "ta"}, "oddball"]}
|
|
result = mask_secret_fields(config, SCHEMA_PROPS)
|
|
assert result["accounts"][0] == {"name": "a", "token": ""}
|
|
assert result["accounts"][1] == "oddball"
|
|
|
|
def test_non_dict_schema_property_tolerated(self):
|
|
assert mask_secret_fields({"x": 1}, {"x": "bogus"}) == {"x": 1}
|
|
|
|
|
|
class TestMaskAllSecretValues:
|
|
def test_real_values_replaced_with_bullets(self):
|
|
assert mask_all_secret_values({"key": "abc"}) == {"key": "••••••••"}
|
|
|
|
def test_placeholders_preserved(self):
|
|
# YOUR_* placeholders must survive so the UI can show "not set".
|
|
result = mask_all_secret_values({"key": "YOUR_API_KEY_HERE"})
|
|
assert result == {"key": "YOUR_API_KEY_HERE"}
|
|
|
|
def test_empty_and_none_preserved(self):
|
|
assert mask_all_secret_values({"a": "", "b": None}) == {"a": "", "b": None}
|
|
|
|
def test_recurses_into_nested_dicts(self):
|
|
result = mask_all_secret_values({"plugin": {"token": "t", "empty": ""}})
|
|
assert result == {"plugin": {"token": "••••••••", "empty": ""}}
|
|
|
|
def test_non_string_real_values_masked(self):
|
|
assert mask_all_secret_values({"port": 8080}) == {"port": "••••••••"}
|
|
|
|
|
|
class TestRemoveEmptySecrets:
|
|
def test_strips_empty_string(self):
|
|
assert remove_empty_secrets({"a": "", "b": "real"}) == {"b": "real"}
|
|
|
|
def test_strips_whitespace_only(self):
|
|
assert remove_empty_secrets({"a": " "}) == {}
|
|
|
|
def test_strips_none(self):
|
|
assert remove_empty_secrets({"a": None}) == {}
|
|
|
|
def test_prunes_empty_nested_dicts(self):
|
|
assert remove_empty_secrets({"plugin": {"token": ""}}) == {}
|
|
|
|
def test_keeps_nested_real_values(self):
|
|
result = remove_empty_secrets({"plugin": {"token": "t", "empty": ""}})
|
|
assert result == {"plugin": {"token": "t"}}
|
|
|
|
def test_keeps_falsey_non_string_values(self):
|
|
# 0 and False are neither None nor blank strings — they are kept.
|
|
assert remove_empty_secrets({"a": 0, "b": False}) == {"a": 0, "b": False}
|