Files
LEDMatrix/test/plugins/conftest.py
T
d6c5f97c13 Test suite overhaul + fixes for the three bugs it uncovered (#441)
* ci: run the whole test tree and make the plugin-safety job assert something real

The unit-tests CI job ran an explicit 24-file allowlist that had rotted:
63 of 90 test files (display, vegas, store manager, web API, web_interface)
never ran on a PR. The job now runs all of test/ (minus test/plugins, which
the plugin-safety job owns) so new test files are enrolled by default and
any exclusion needs a visible, commented --ignore.

The plugin-safety job was a green no-op: plugins/ is empty in CI, so every
test skipped with 'Manifest not found'. It now renders a bundled
deterministic fixture plugin (test/fixtures/plugins/ci-fixture-plugin,
golden images included for all 8 default sizes) via LEDMATRIX_PLUGINS_DIR,
and sets LEDMATRIX_REQUIRE_PLUGINS=1 so discovering zero plugins fails
loudly instead of skipping green. The per-plugin suites document that they
target dev machines with real plugins installed.

Coverage is now measured and enforced in exactly one place — the CI
unit-tests step (--cov=src --cov=web_interface --cov-fail-under=45, from a
measured 47% baseline). pytest.ini previously declared --cov-fail-under=30
but CI always passed --no-cov, so the gate had never run anywhere; local
pytest is now coverage-free and fast.

Enabling the 63 unenrolled files surfaced three cases of test rot, fixed
here: test_display_controller_vegas_tick.py could not collect without the
hardware rgbmatrix module (now uses the emulator convention), the
state-reconciliation unrecoverable-cache tests broke when production added
the is_plugin_uninstalled tombstone check (bare Mock returned truthy),
and test_get_system_status assumed the optional psutil dependency
(now installed via requirements-test.txt and guarded by importorskip).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

* test: replace can't-fail tests with real assertions

test_font_manager.py was 5 of 6 tests shaped as 'try: call(); assert True /
except: assert True' — running in CI while unable to fail on any
regression. Rewritten against the real FontManager API and the bundled
assets/fonts: returned font types, cache-hit identity, distinct entries per
size, default-font fallback for unknown families and corrupt files
(recorded in failed_loads), BDF native-size reading, text measurement, and
cache lifecycle.

test_display_manager.py's test_draw_text ended in 'assert True'; it now
renders onto a known-black canvas and asserts pixels were actually lit —
which required un-breaking the fixture's freetype MagicMock so draw_text's
isinstance check doesn't silently swallow the draw.

test_display_controller.py carried a permanently-skipped test whose skip
reason already declared it redundant; deleted.

Both display test files now set EMULATOR=true before importing
display_manager (the same convention as test_display_dirty_tracking.py) so
they collect standalone instead of depending on which test module imports
display_manager first.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

* test: cover the untested fragile logic (compatibility gate, secrets, config merges, durations, skin cards)

New unit tests for pure or filesystem-only logic that previously had zero
direct coverage:

- test_compatibility.py: the semver install gate (parse_semver suffix
  handling, every range operator, TRUSTWORTHY_FLOOR behavior for cores
  reporting untrustworthy versions, 'more restrictive wins', and the
  malformed-manifest shapes that used to raise).
- test/web_interface/test_secret_helpers.py: the canonical x-secret
  helpers — find/separate/mask/remove, array-item secrets, no input
  mutation, and a separate->recombine round-trip.
- test/web_interface/test_api_v3_helpers.py: the module-level helpers
  behind the plugin config save endpoint (_is_plugin_update_available,
  _coerce_to_bool including the int==1 quirk, deep_merge including its
  shared-subtree shallowness, _parse_form_value, dotted-key-aware
  _get_schema_property/_set_nested_value).
- test_base_plugin_duration.py: get_display_duration's full coercion
  ladder (instance attr -> config -> 15.0), including the bool-is-int
  quirk where display_duration=True means one second.
- test_config_manager_secrets.py: the secrets round-trip — deep-merge on
  load, strip on save, group pruning, the load fast path — and two
  characterized sharp edges marked SUSPECTED BUG: an unreadable secrets
  file at save time writes secrets into config.json in plaintext, and a
  same-mtime-same-size content swap is served stale.
- test_schema_manager_merge.py: merge_with_defaults branch behavior (None
  replacement vs falsey preservation, dict-vs-scalar mismatches, arrays
  replaced wholesale, defaults never mutated).
- test_skin_system.py (extended): render_skin_card shares _render_game's
  3-strike counter but never resets it on success — the asymmetry is
  pinned in both directions, along with card fallthrough and the disable
  interaction between the two paths.

Suspected bugs are characterized, not fixed — each carries a comment so a
future behavior change is deliberate rather than accidental.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

* test: add drift guards for cross-file contracts

Three guard suites that pin contracts spanning multiple files, where one
side changing unilaterally breaks the other silently:

- test_version_comparison_consistency.py: the repo's four version
  comparators (compatibility.parse_semver, api_v3's packaging-based
  _is_plugin_update_available, store_manager update_plugin's raw string
  equality, skin_runtime._major) answer differently on the same inputs.
  A table pins each one's verdict; update_plugin is driven through its
  real code path to show the SUSPECTED BUGs: 'v1.2.0' vs '1.2.0'
  triggers a full reinstall the UI calls unnecessary, and a locally-ahead
  plugin gets downgraded. A pairwise-ordering check keeps parse_semver
  agreeing with packaging on plain X.Y.Z.
- test/web_interface/test_secret_separation_parity.py: api_v3.py carries
  three inline copies of find_secret_fields/separate_secrets that lack
  the canonical module's array-item support. The copy count is asserted
  exact (it may only go down; new copies must import
  src/web_interface/secret_helpers), the missing-array-support gap is
  asserted so it can't grow silently, and the canonical behavior that
  migration will adopt is documented executably.
- test_discovery_path_contract.py: the three 'where is plugin X'
  resolvers (PluginManager discovery, StoreManager._find_plugin_path,
  SchemaManager.get_schema_path) agree on the configured directory, and
  their divergent fallback chains are characterized. Also pins the
  .standalone-backup- naming contract shared by store rollback and
  discovery, and _resolve_skin_target's path-traversal rejection.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

* test: address review feedback — fixture lifecycle, test names, ClassVar

- ci-fixture-plugin: call display_manager.clear() before rendering (per
  plugin guidelines — the fixture should model a well-behaved plugin),
  add a class docstring, and document why Pillow is deliberately not
  pinned in its requirements.txt (core dependency; harness installs
  nothing).
- Rename two tests whose names contradicted their assertions:
  test_unparseable_core_version_is_compatible ->
  test_unparseable_core_with_high_floor_is_blocked, and
  test_unreadable_secrets_file... -> test_corrupt_secrets_file...
- Annotate TestGetSchemaProperty.SCHEMA as ClassVar (RUF012).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

* ci: allow manual test.yml runs via workflow_dispatch

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

* fix: unify version comparison, refuse secret-leaking saves, reset skin strikes on card success

Fixes the three suspected bugs this PR's characterization tests pinned,
flipping those tests to assert the corrected behavior:

- plugins/store: ONE shared update comparator. New
  compatibility.is_update_available() (PEP 440 via packaging) is now used
  by both the web UI's update badge (api_v3._is_plugin_update_available
  is a thin alias) and store_manager.update_plugin's reinstall decision.
  Previously update_plugin used raw string equality: 'v1.2.0' vs '1.2.0'
  triggered a full reinstall the UI called unnecessary, and a locally-
  ahead plugin (2.0.0 installed, registry 1.9.0) was silently DOWNGRADED.
  Now equivalent spellings skip the reinstall and locally-ahead versions
  are never downgraded; unparseable versions still reconcile by
  reinstalling from the registry.

- config: save_config and save_config_atomic now refuse (ConfigError)
  when config_secrets.json exists but cannot be loaded. Both previously
  proceeded without stripping, writing the merged secrets into
  config.json in plaintext. The shared _load_secrets_for_save() helper
  raises with an actionable message instead; a missing secrets file is
  still fine (nothing to strip), and _migrate_config's catch-all keeps
  boot resilient.

- skins: render_skin_card resets _skin_failures on both success paths
  (vegas card returned, or mode renderer handled), mirroring
  _render_game. Transient card failures no longer accumulate across a
  session until they permanently disable a working skin.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

* fix: harden shared comparator edges from review

- is_update_available: reject truthy non-string versions (a malformed
  manifest can carry a number; packaging raises TypeError on those) by
  surfacing the mismatch instead of raising.
- store_manager.update_plugin: drop the truthiness gate around the
  comparator so a missing version on either side follows the shared
  'no update' verdict, keeping the store consistent with the UI badge;
  a missing manifest still uses the reinstall recovery path.
- config_manager._load_secrets_for_save: catch only expected read/parse
  failures (OSError/ValueError/RecursionError) so implementation bugs
  propagate as themselves, and log with traceback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-07 10:17:30 -04:00

140 lines
3.9 KiB
Python

"""
Pytest fixtures for plugin integration tests.
"""
import pytest
import os
import sys
import json
from pathlib import Path
from unittest.mock import MagicMock, Mock
from typing import Any, Dict
# Add project root to path
project_root = Path(__file__).parent.parent.parent
if str(project_root) not in sys.path:
sys.path.insert(0, str(project_root))
# Set emulator mode
os.environ['EMULATOR'] = 'true'
@pytest.fixture
def plugins_dir() -> Path:
"""Get the plugins directory path.
Honors LEDMATRIX_PLUGINS_DIR (first entry) when set — the same override
test_plugin_matrix.py uses, so CI can point every plugin suite at the
bundled fixture plugins. Otherwise checks plugins/ first, then falls
back to plugin-repos/ for monorepo development environments.
"""
env = os.environ.get('LEDMATRIX_PLUGINS_DIR')
if env:
first = env.split(os.pathsep)[0]
if first:
return Path(first)
plugins_path = project_root / 'plugins'
plugin_repos_path = project_root / 'plugin-repos'
# Prefer plugins/ if it has actual plugin directories
if plugins_path.exists():
try:
has_plugins = any(
p for p in plugins_path.iterdir()
if p.is_dir() and not p.name.startswith('.')
)
if has_plugins:
return plugins_path
except PermissionError:
pass
if plugin_repos_path.exists():
return plugin_repos_path
return plugins_path
@pytest.fixture
def mock_display_manager() -> Any:
"""Create a mock DisplayManager for plugin tests."""
mock = MagicMock()
mock.width = 128
mock.height = 32
mock.clear = Mock()
mock.draw_text = Mock()
mock.draw_image = Mock()
mock.update_display = Mock()
mock.get_font = Mock(return_value=None)
# Some plugins access matrix.width/height
mock.matrix = MagicMock()
mock.matrix.width = 128
mock.matrix.height = 32
return mock
@pytest.fixture
def mock_cache_manager() -> Any:
"""Create a mock CacheManager for plugin tests."""
mock = MagicMock()
mock._memory_cache = {}
def mock_get(key: str, max_age: int = 300) -> Any:
return mock._memory_cache.get(key)
def mock_set(key: str, data: Any, ttl: int = None) -> None:
mock._memory_cache[key] = data
def mock_clear(key: str = None) -> None:
if key:
mock._memory_cache.pop(key, None)
else:
mock._memory_cache.clear()
mock.get = Mock(side_effect=mock_get)
mock.set = Mock(side_effect=mock_set)
mock.clear = Mock(side_effect=mock_clear)
return mock
@pytest.fixture
def mock_plugin_manager() -> Any:
"""Create a mock PluginManager for plugin tests."""
mock = MagicMock()
mock.plugins = {}
mock.plugin_manifests = {}
return mock
@pytest.fixture
def base_plugin_config() -> Dict[str, Any]:
"""Base configuration for plugins."""
return {
'enabled': True,
'update_interval': 300
}
def load_plugin_manifest(plugin_id: str, plugins_dir: Path) -> Dict[str, Any]:
"""Load plugin manifest.json."""
manifest_path = plugins_dir / plugin_id / 'manifest.json'
if not manifest_path.exists():
pytest.skip(f"Manifest not found for {plugin_id}")
with open(manifest_path, 'r') as f:
return json.load(f)
def get_plugin_config_schema(plugin_id: str, plugins_dir: Path) -> Dict[str, Any]:
"""Load plugin config_schema.json if it exists."""
schema_path = plugins_dir / plugin_id / 'config_schema.json'
if schema_path.exists():
with open(schema_path, 'r') as f:
return json.load(f)
return None
@pytest.fixture
def visual_display_manager() -> Any:
"""Create a VisualTestDisplayManager that renders real pixels for visual testing."""
from src.plugin_system.testing import VisualTestDisplayManager
return VisualTestDisplayManager(width=128, height=32)