Files
LEDMatrix/src/plugin_system/testing/bounds_display_manager.py
T
313e35a98f Add cross-size/cross-screen plugin safety harness (#361)
* feat(testing): add cross-size/cross-screen plugin safety harness

Render every plugin across all supported matrix sizes (64x32, 128x32,
128x64, 256x32) and every declared screen, failing on crashes, content
drawn past the panel edge, or visual drift vs committed golden images.

- BoundsCheckingDisplayManager: oversized-canvas overflow detection
- harness.py: multi-size/multi-screen render engine + golden compare
- scripts/check_plugin.py: CLI (functional+bounds, --out-dir, --update-golden,
  --freeze-time); render_plugin.py refactored onto shared loading helpers
- test/plugins/test_harness.py + test_plugin_matrix.py (parametrized,
  honors per-plugin test/harness.json; skips when no plugins present)
- MockCacheManager.cache_dir so cache-dir-using plugins load headlessly
- .github/workflows/test.yml + docs/plugin-safety-harness.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(testing): address PR review feedback on plugin safety harness

- check_plugin: friendly error for non-numeric --sizes; reject non-object
  --config / --mock-data JSON; sanitize plugin mode before using as a
  filename; stop --update-golden from masking crash/overflow failures
- bounds_display_manager: pad the canvas out to the largest supported panel
  (not a fixed 16px) so far-overshoot coordinates are caught, not clipped
- harness: merge config_schema defaults inside render_plugin_matrix; surface
  update() failures as a non-fatal warning + result field instead of a debug
  log; sanitize mode in golden_path
- loading: fail fast when harness.json references a missing mock_data fixture
- mocks: clean up the per-instance temp cache dir via weakref.finalize
- test_plugin_matrix: add a discovery guard that fails when
  LEDMATRIX_REQUIRE_PLUGINS=1 but none found (still skips locally); type hints
- bound test deps with upper version pins for deterministic CI

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(testing): render plugins across arbitrary panel sizes, not a fixed list

Addresses maintainer feedback that there is no canonical set of supported
panel sizes — a build can be any size/configuration (square, 2x2, 4x4, 8x2,
long strips, tall stacks).

- sizes.py: SUPPORTED_SIZES -> DEFAULT_TEST_SIZES (back-compat alias kept),
  reframed as a representative SAMPLE of real panel-grid arrangements rather
  than an authoritative list; add parse_size_token / coerce_sizes /
  resolve_test_sizes helpers
- sizes are now fully overridable: LEDMATRIX_TEST_SIZES env (global, e.g. test
  on your exact hardware) > per-plugin harness.json "sizes" > default sample;
  CLI --sizes unchanged
- bounds_display_manager: pad the canvas to the largest panel IN THE CURRENT
  RUN (via overflow_extent) instead of a hardcoded max, so cross-size overflow
  detection scales to whatever sizes a run uses
- harness: compute per-run extent and thread it into the bounds manager
- tests: arbitrary-shape + size-parsing/precedence coverage
- docs: rewrite "Supported sizes" -> "Sizes: a sample, not a fixed list"

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(testing): fail the harness on non-connectivity update() errors

Addresses the remaining review thread: recording every update() exception as a
non-fatal warning still let a real update() regression pass green as long as
display() survived. Now update() failures are classified — a tolerated set of
connectivity errors (ConnectionError/TimeoutError/socket/ssl/urllib/http/
requests) is recorded non-fatally (expected with no network in CI), while any
other exception is treated as a genuine bug and fails that render.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci(security): pin actions to SHAs and disable checkout credential persistence

Addresses the CodeRabbit/zizmor workflow-hardening finding: pin
actions/checkout and actions/setup-python to full commit SHAs and set
persist-credentials: false on checkout to reduce supply-chain and
token-exposure risk.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(testing): validate positive sizes; narrow requests import except

Two review findings:
- sizes.py: parse_size_token / coerce_sizes now reject non-positive
  dimensions (0x32, -64x32) with a clear message instead of passing invalid
  sizes downstream (CodeRabbit).
- harness.py: the optional `requests` import now catches ImportError
  specifically and logs instead of `except Exception: pass`, clearing the
  Codacy medium "Try, Except, Pass" (harness.py L52) and Ruff S110/BLE001.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 14:32:52 -04:00

130 lines
5.4 KiB
Python

"""
Bounds-checking display manager.
A VisualTestDisplayManager that draws onto an oversized canvas (the declared
panel size plus a right/bottom margin) while still reporting the declared size
to the plugin. Content that a plugin draws past the right or bottom edge lands
in the margin instead of being silently clipped by PIL, so the harness can
detect overflow — the classic symptom of hardcoded coordinates or fonts/icons
that don't scale down to a smaller panel.
Limitations (documented on purpose):
- Overflow past the LEFT or TOP edge (negative coordinates) is still clipped by
PIL and not detected here. The dominant real-world breakage is content that is
too wide/tall for a smaller panel, which this catches.
- BDF text is clipped to the declared bounds by the parent's bitmap drawer, so
BDF overflow is not flagged. Golden-image regression covers those plugins.
- If a plugin replaces the canvas with its own image (display_manager.image = ...),
the margin can't be measured and overflow is reported as undetermined (None).
"""
from typing import Optional, Tuple
from .sizes import DEFAULT_TEST_SIZES
from .visual_display_manager import VisualTestDisplayManager, _MatrixProxy
# Smallest extra band kept on the right/bottom so a few pixels of overflow are
# still visible even on the largest panel in a run.
_BASE_MARGIN = 16
# Fallback overflow reference when a caller doesn't pass one: the largest shape
# in the default sample. We extend every (smaller) canvas out to at least this
# size so content drawn at a coordinate meant for a bigger build — e.g. x=200 on
# a 64-wide panel — lands in the padded region and is flagged, instead of being
# clipped off-canvas and read as a false pass.
_DEFAULT_EXTENT_WIDTH = max(w for w, _ in DEFAULT_TEST_SIZES)
_DEFAULT_EXTENT_HEIGHT = max(h for _, h in DEFAULT_TEST_SIZES)
class BoundsCheckingDisplayManager(VisualTestDisplayManager):
"""Detects drawing that overflows the declared panel size."""
# Kept for backwards compatibility; real padding is computed per-axis below.
MARGIN = _BASE_MARGIN
def __init__(self, width: int = 128, height: int = 32,
overflow_extent: Optional[Tuple[int, int]] = None):
self._declared_width = int(width)
self._declared_height = int(height)
# Pad the canvas out to at least `overflow_extent` (the largest panel
# this run cares about) plus a base margin, so coordinates meant for a
# bigger build are caught — not clipped — when rendering a smaller panel.
# Defaults to the largest shape in the sample when no run is known.
ext_w, ext_h = overflow_extent or (_DEFAULT_EXTENT_WIDTH, _DEFAULT_EXTENT_HEIGHT)
self._canvas_width = max(self._declared_width, int(ext_w)) + _BASE_MARGIN
self._canvas_height = max(self._declared_height, int(ext_h)) + _BASE_MARGIN
# Parent builds the (oversized) backing canvas + fonts.
super().__init__(self._canvas_width, self._canvas_height)
# Plugins must see the DECLARED size, not the padded canvas size.
self.matrix = _MatrixProxy(self._declared_width, self._declared_height)
# -- declared dimensions (override parent's image-derived properties) --
@property
def width(self) -> int:
return self._declared_width
@property
def height(self) -> int:
return self._declared_height
@property
def display_width(self) -> int:
return self._declared_width
@property
def display_height(self) -> int:
return self._declared_height
# -- overflow detection --
def _canvas_is_padded(self) -> bool:
return self.image.size == (self._canvas_width, self._canvas_height)
def check_overflow(self) -> Optional[Tuple[int, int, int, int]]:
"""Bounding box (in full-canvas coords) of any drawing beyond the
declared panel, or None if nothing overflowed / undetermined."""
if not self._canvas_is_padded():
return None
exp_w = self._canvas_width
exp_h = self._canvas_height
boxes = []
right = self.image.crop((self._declared_width, 0, exp_w, exp_h)).getbbox()
if right:
boxes.append((right[0] + self._declared_width, right[1],
right[2] + self._declared_width, right[3]))
bottom = self.image.crop((0, self._declared_height, exp_w, exp_h)).getbbox()
if bottom:
boxes.append((bottom[0], bottom[1] + self._declared_height,
bottom[2], bottom[3] + self._declared_height))
if not boxes:
return None
return (
min(b[0] for b in boxes), min(b[1] for b in boxes),
max(b[2] for b in boxes), max(b[3] for b in boxes),
)
# -- snapshot/image accessors return the cropped, true-panel image --
def declared_image(self):
"""The visible panel: the canvas cropped to the declared size."""
if self._canvas_is_padded():
return self.image.crop((0, 0, self._declared_width, self._declared_height))
return self.image
def save_snapshot(self, path: str) -> None:
self.declared_image().save(path, format='PNG')
def get_image(self):
return self.declared_image()
def get_image_base64(self) -> str:
import base64
import io
buffer = io.BytesIO()
self.declared_image().save(buffer, format='PNG')
return base64.b64encode(buffer.getvalue()).decode('utf-8')