Files
LEDMatrix/src/plugin_system/testing/mocks.py
Ron Pierce 313e35a98f Add cross-size/cross-screen plugin safety harness (#361)
* feat(testing): add cross-size/cross-screen plugin safety harness

Render every plugin across all supported matrix sizes (64x32, 128x32,
128x64, 256x32) and every declared screen, failing on crashes, content
drawn past the panel edge, or visual drift vs committed golden images.

- BoundsCheckingDisplayManager: oversized-canvas overflow detection
- harness.py: multi-size/multi-screen render engine + golden compare
- scripts/check_plugin.py: CLI (functional+bounds, --out-dir, --update-golden,
  --freeze-time); render_plugin.py refactored onto shared loading helpers
- test/plugins/test_harness.py + test_plugin_matrix.py (parametrized,
  honors per-plugin test/harness.json; skips when no plugins present)
- MockCacheManager.cache_dir so cache-dir-using plugins load headlessly
- .github/workflows/test.yml + docs/plugin-safety-harness.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(testing): address PR review feedback on plugin safety harness

- check_plugin: friendly error for non-numeric --sizes; reject non-object
  --config / --mock-data JSON; sanitize plugin mode before using as a
  filename; stop --update-golden from masking crash/overflow failures
- bounds_display_manager: pad the canvas out to the largest supported panel
  (not a fixed 16px) so far-overshoot coordinates are caught, not clipped
- harness: merge config_schema defaults inside render_plugin_matrix; surface
  update() failures as a non-fatal warning + result field instead of a debug
  log; sanitize mode in golden_path
- loading: fail fast when harness.json references a missing mock_data fixture
- mocks: clean up the per-instance temp cache dir via weakref.finalize
- test_plugin_matrix: add a discovery guard that fails when
  LEDMATRIX_REQUIRE_PLUGINS=1 but none found (still skips locally); type hints
- bound test deps with upper version pins for deterministic CI

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(testing): render plugins across arbitrary panel sizes, not a fixed list

Addresses maintainer feedback that there is no canonical set of supported
panel sizes — a build can be any size/configuration (square, 2x2, 4x4, 8x2,
long strips, tall stacks).

- sizes.py: SUPPORTED_SIZES -> DEFAULT_TEST_SIZES (back-compat alias kept),
  reframed as a representative SAMPLE of real panel-grid arrangements rather
  than an authoritative list; add parse_size_token / coerce_sizes /
  resolve_test_sizes helpers
- sizes are now fully overridable: LEDMATRIX_TEST_SIZES env (global, e.g. test
  on your exact hardware) > per-plugin harness.json "sizes" > default sample;
  CLI --sizes unchanged
- bounds_display_manager: pad the canvas to the largest panel IN THE CURRENT
  RUN (via overflow_extent) instead of a hardcoded max, so cross-size overflow
  detection scales to whatever sizes a run uses
- harness: compute per-run extent and thread it into the bounds manager
- tests: arbitrary-shape + size-parsing/precedence coverage
- docs: rewrite "Supported sizes" -> "Sizes: a sample, not a fixed list"

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(testing): fail the harness on non-connectivity update() errors

Addresses the remaining review thread: recording every update() exception as a
non-fatal warning still let a real update() regression pass green as long as
display() survived. Now update() failures are classified — a tolerated set of
connectivity errors (ConnectionError/TimeoutError/socket/ssl/urllib/http/
requests) is recorded non-fatally (expected with no network in CI), while any
other exception is treated as a genuine bug and fails that render.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci(security): pin actions to SHAs and disable checkout credential persistence

Addresses the CodeRabbit/zizmor workflow-hardening finding: pin
actions/checkout and actions/setup-python to full commit SHAs and set
persist-credentials: false on checkout to reduce supply-chain and
token-exposure risk.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(testing): validate positive sizes; narrow requests import except

Two review findings:
- sizes.py: parse_size_token / coerce_sizes now reject non-positive
  dimensions (0x32, -64x32) with a clear message instead of passing invalid
  sizes downstream (CodeRabbit).
- harness.py: the optional `requests` import now catches ImportError
  specifically and logs instead of `except Exception: pass`, clearing the
  Codacy medium "Try, Except, Pass" (harness.py L52) and Ruff S110/BLE001.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 14:32:52 -04:00

193 lines
6.0 KiB
Python

"""
Mock objects for plugin testing.
Provides mock implementations of display_manager, cache_manager, config_manager,
and plugin_manager for use in plugin unit tests.
"""
from typing import Dict, Any, Optional
from PIL import Image
class MockDisplayManager:
"""Mock display manager for testing."""
def __init__(self, width: int = 128, height: int = 32):
self.width = width
self.display_width = width
self.height = height
self.display_height = height
self.image = Image.new('RGB', (width, height), color=(0, 0, 0))
self.clear_called = False
self.update_called = False
self.draw_calls = []
def clear(self):
"""Clear the display."""
self.clear_called = True
self.image = Image.new('RGB', (self.width, self.height), color=(0, 0, 0))
def update_display(self):
"""Update the display."""
self.update_called = True
def draw_text(self, text: str, x: int, y: int, color: tuple = (255, 255, 255), font=None):
"""Draw text on the display."""
self.draw_calls.append({
'type': 'text',
'text': text,
'x': x,
'y': y,
'color': color,
'font': font
})
def draw_image(self, image: Image.Image, x: int, y: int):
"""Draw an image on the display."""
self.draw_calls.append({
'type': 'image',
'image': image,
'x': x,
'y': y
})
def reset(self):
"""Reset mock state."""
self.clear_called = False
self.update_called = False
self.draw_calls = []
self.image = Image.new('RGB', (self.width, self.height), color=(0, 0, 0))
class MockCacheManager:
"""Mock cache manager for testing."""
def __init__(self):
import shutil
import tempfile
import weakref
self._cache: Dict[str, Any] = {}
self._cache_timestamps: Dict[str, float] = {}
self.get_calls = []
self.set_calls = []
self.delete_calls = []
# Real temp dir for plugins that write/read files under cache_dir.
# Registered for cleanup so each mock instance doesn't leak a tmp dir.
self.cache_dir = tempfile.mkdtemp(prefix="ledmatrix-mock-cache-")
self._finalizer = weakref.finalize(
self, shutil.rmtree, self.cache_dir, ignore_errors=True)
def cleanup(self) -> None:
"""Remove the temp cache directory created for this instance."""
self._finalizer()
def get(self, key: str, max_age: Optional[float] = None) -> Optional[Any]:
"""Get a value from cache."""
import time
self.get_calls.append({'key': key, 'max_age': max_age})
if key not in self._cache:
return None
if max_age is not None:
timestamp = self._cache_timestamps.get(key, 0)
if time.time() - timestamp > max_age:
return None
return self._cache.get(key)
def set(self, key: str, value: Any, ttl: Optional[float] = None) -> None:
"""Set a value in cache."""
import time
self.set_calls.append({'key': key, 'value': value, 'ttl': ttl})
self._cache[key] = value
self._cache_timestamps[key] = time.time()
def delete(self, key: str) -> None:
"""Delete a value from cache."""
self.delete_calls.append(key)
if key in self._cache:
del self._cache[key]
if key in self._cache_timestamps:
del self._cache_timestamps[key]
def reset(self):
"""Reset mock state."""
self._cache.clear()
self._cache_timestamps.clear()
self.get_calls = []
self.set_calls = []
self.delete_calls = []
class MockConfigManager:
"""Mock config manager for testing."""
def __init__(self, config: Optional[Dict[str, Any]] = None):
self._config = config or {}
self.load_config_calls = []
self.save_config_calls = []
def load_config(self) -> Dict[str, Any]:
"""Load configuration."""
self.load_config_calls.append({})
return self._config.copy()
def save_config(self, config: Dict[str, Any]) -> None:
"""Save configuration."""
self.save_config_calls.append(config)
self._config = config.copy()
def get_config(self, key: str, default: Any = None) -> Any:
"""Get a config value."""
return self._config.get(key, default)
def set_config(self, key: str, value: Any) -> None:
"""Set a config value."""
self._config[key] = value
def reset(self):
"""Reset mock state."""
self._config = {}
self.load_config_calls = []
self.save_config_calls = []
class MockPluginManager:
"""Mock plugin manager for testing."""
def __init__(self):
self.plugins: Dict[str, Any] = {}
self.plugin_manifests: Dict[str, Dict] = {}
self.get_plugin_calls = []
self.get_all_plugins_calls = []
def get_plugin(self, plugin_id: str) -> Optional[Any]:
"""Get a plugin instance."""
self.get_plugin_calls.append(plugin_id)
return self.plugins.get(plugin_id)
def get_all_plugins(self) -> Dict[str, Any]:
"""Get all plugin instances."""
self.get_all_plugins_calls.append({})
return self.plugins.copy()
def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""Get plugin information."""
manifest = self.plugin_manifests.get(plugin_id, {})
plugin = self.plugins.get(plugin_id)
if plugin:
manifest['loaded'] = True
manifest['runtime_info'] = getattr(plugin, 'get_info', lambda: {})()
else:
manifest['loaded'] = False
return manifest
def reset(self):
"""Reset mock state."""
self.plugins.clear()
self.plugin_manifests.clear()
self.get_plugin_calls = []
self.get_all_plugins_calls = []