Compare commits

..
Author SHA1 Message Date
ChuckBuildsandClaude Opus 5 b32563ca31 ci: run the new suites, and check tag/version agreement at release time
These were split out of #428/#429 because the token pushing them lacked the
`workflow` scope. Folding them in here rather than opening a stacked PR --
#429 was merged into its stacked base after that base had already been
squash-merged, so its content never reached main, and one such near-miss is
enough.

All three enrolled suites exist on this branch: test_version_consistency.py
came with #428 and is on main; the other two arrive with the commits above.
Enrolling them in a separate PR would have either raced with this one on
test.yml or briefly pointed CI at files main did not have.

- test.yml: enroll test_version_consistency, test_plugin_compatibility_gate
  and test_install_preserves_existing in the core unit job. Until now these
  32 tests existed but nothing ran them automatically.

- release-version-check.yml: run scripts/check_release_version.py on pushed
  v* tags and published releases, plus workflow_dispatch so a tag can be
  checked *before* it is created. No dependencies -- it reads src/__init__.py
  and CHANGELOG.md only.

Verified: both workflow files parse, and the release check still passes for
v3.2.0 against this tree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5
2026-08-03 16:24:09 -04:00
ChuckBuildsandClaude Opus 5 1bbf93484d fix(store): serialize concurrent installs, and make the lock reentrant
Second bug found while validating the previous commit on hardware.

install_plugin's new set-aside/restore had no lock. The web UI runs Flask
threaded, so a double-clicked Install button gives two threads the same
plugin_id; interleaved, one thread's restore deletes the other's freshly
installed copy. _reinstall_with_rollback already guards exactly this with a
per-plugin lock, and install_plugin needs the same one.

Taking that lock naively deadlocks. _reinstall_with_rollback holds it across
its call to install_plugin, and threading.Lock is not reentrant -- so the
request thread hangs forever on the standard monorepo update path
(update_plugin -> _reinstall_with_rollback -> install_plugin), which is to say
on every plugin update. Verified by reverting to a plain Lock: the regression
test times out after 10s instead of passing.

The per-plugin locks are now RLocks, and install_plugin holds one for its
whole set-aside/install/restore sequence.

Verified on devpi (Pi, Python 3.13.5, real registry and network):
- update_plugin on an up-to-date plugin: True in 5.4s
- update_plugin forced through the full reinstall-with-rollback path:
  True in 13.1s, correct version restored, old copy replaced, no backup
  directories left behind
- install -> reinstall-over-existing -> failed-reinstall-restores: all pass
  against real downloads
- 22 plugins load, no tracebacks, web API and UI 200, steady-state journal
  50 lines/min

791 core unit tests pass, including 2 new concurrency tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5
2026-08-03 16:17:28 -04:00
ChuckBuildsandClaude Opus 5 dd46b7f862 fix(store): a failed install must not destroy the plugin it replaced
Found while validating the compatibility gate. `_install_plugin_impl` deletes
the existing plugin directory *before* downloading, so any failure after that
point leaves the user with nothing. `_reinstall_with_rollback` protects the
update path exactly this way; a direct `install_plugin` had no equivalent.

The gate made this reachable in a new way: a plugin whose declared floor
exceeds the running core is now refused *after* the old copy is already gone.
Floors are hand-written and can be over-declared, so the refusal could remove
a plugin that had been working fine on that core.

install_plugin is now a thin wrapper that renames any existing install aside,
delegates to _install_plugin_impl, and restores it on failure -- including
when the implementation raises, which is re-raised after the restore. It is a
pass-through when nothing is installed and when called from
_reinstall_with_rollback, which has already moved the old copy aside; a test
pins that so the two mechanisms cannot start nesting.

The aside name embeds '.standalone-backup-' because
plugin_manager._scan_directory_for_plugins keys on exactly that substring to
skip backups. A different name would have made the backup discoverable as a
duplicate plugin; a test pins that too.

789 core unit tests pass, including 7 new ones.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5
2026-08-03 16:17:28 -04:00
ChuckBuildsandClaude Opus 5 2d531c2cb6 feat(store): refuse to install a plugin that needs a newer core
`ledmatrix_min_version` was decoration. The loader logged an advisory warning
and continued; the store never compared the core version at all, so a routine
"update" delivered a plugin that could not run. That is the gap phase B6 (the
sports-unification sunset) cannot be done over: deleting a plugin's bundled
fallback while nothing enforces the floor hands un-updated users a scoreboard
that raises ModuleNotFoundError at load and is reported only as one line in
the journal.

The gate lives in install_plugin, after the manifest is on disk and before
dependencies are installed. That is the earliest knowable point -- the
registry carries no compatibility field, so the floor is not visible until
the files are down -- and it is also the chokepoint: _reinstall_with_rollback
calls install_plugin, so a refused *update* restores the version the user
already had, for free.

Floor resolution and the comparison move to src/plugin_system/compatibility.py,
shared with the loader so the two cannot drift. Both read all four spellings
published manifests use, including the deprecated `ledmatrix_min`.

Refusal requires evidence. An undeclared floor, an unparseable version on
either side, or a core below TRUSTWORTHY_FLOOR (2.0.0) all allow the install.
That last one is deliberate and load-bearing: the v3.1.0 release reports
__version__ = "1.0.0" while nearly every published manifest floors at 2.0.0,
so a strict gate would lock those users out of the plugin store entirely --
much worse than the problem being solved. They stay unprotected until they
update the core, which is also what fixes their version string.

Verified: 782 core unit tests pass, including 25 new ones and the existing
loader-warning suite unchanged (the refactor is behavior-preserving). The
install tests drive the real install_plugin path with the download stubbed --
the allow and refuse cases differ only in the declared floor, so the refusal
is demonstrably the gate and not an earlier bail-out.

Follow-ups, deliberately not in this PR: surfacing the reason in the store UI
rather than only the log, and publishing the floor in plugins.json so the
store can refuse before downloading.

Phase B4 in docs/SPORTS_UNIFICATION.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5
2026-08-03 16:17:28 -04:00
4 changed files with 2 additions and 182 deletions
+1 -2
View File
@@ -79,5 +79,4 @@ jobs:
test/test_sports_scroll.py \ test/test_sports_scroll.py \
test/test_version_consistency.py \ test/test_version_consistency.py \
test/test_plugin_compatibility_gate.py \ test/test_plugin_compatibility_gate.py \
test/test_install_preserves_existing.py \ test/test_install_preserves_existing.py
test/test_core_owned_config_keys.py
-50
View File
@@ -29,21 +29,6 @@ Adoption is deliberately staged: the modules below ship here, plugins adopt them
behind guarded imports, and only then do the bundled copies go away. Nothing in behind guarded imports, and only then do the bundled copies go away. Nothing in
this release changes what an existing plugin loads. this release changes what an existing plugin loads.
**This is also the first release that *enforces* `ledmatrix_min_version`.**
Before it, the floor was advisory — the loader logged a warning and continued,
and the plugin store never compared the core version at all, so an update could
deliver a plugin that could not run. From 3.2.0 the store refuses such an
install. That matters for the sunset rule: a plugin may only delete its bundled
fallback once the cores in the field actually enforce the floor, which means
waiting for 3.2.0 to be widely installed rather than merely released. See
`docs/SPORTS_UNIFICATION.md`, phase B6.
One deliberate exception: a core reporting a version below `2.0.0` is treated as
*unknown* rather than old and is never blocked. The v3.1.0 release ships
`__version__ = "1.0.0"` (the tag was cut before the string was bumped), and
nearly every published manifest floors at `2.0.0` — so blocking on that number
would lock those users out of the plugin store entirely.
### Added ### Added
- `src/element_style.py` — per-element style resolver backing the - `src/element_style.py` — per-element style resolver backing the
`x-style-elements` config-schema extension. Already consumed (behind guarded `x-style-elements` config-schema extension. Already consumed (behind guarded
@@ -86,37 +71,8 @@ would lock those users out of the plugin store entirely.
override point — see `docs/SPORTS_UNIFICATION.md` for where the line falls override point — see `docs/SPORTS_UNIFICATION.md` for where the line falls
and why. and why.
- `src/plugin_system/compatibility.py` — the single place that answers "can this
plugin run on this core?", shared by the loader (advisory, at load time) and
the store (blocking, at install/update time) so the two cannot drift. Reads
every spelling published manifests use, including the deprecated
`versions[].ledmatrix_min`. It does **not** yet evaluate `compatible_versions`,
which is the schema-required field and can express upper bounds; closing that
is tracked in `docs/SPORTS_UNIFICATION.md` before B6.
- `scripts/check_release_version.py` and a `Release version check` workflow —
assert that a tag, the newest CHANGELOG heading and `src.__version__` agree,
on pushed `v*` tags and published releases. Runnable via `workflow_dispatch`
to check a tag *before* creating it. Added because `v3.1.0` was tagged six
weeks before `src/__init__.py` was bumped to match, which is why devices
installed from that release report `1.0.0`.
### Changed ### Changed
- `src/__init__.py` bumped to **3.2.0** — the number the sunset rule keys on. - `src/__init__.py` bumped to **3.2.0** — the number the sunset rule keys on.
- **The plugin store refuses an incompatible install.**
`StoreManager.install_plugin` now checks the downloaded manifest's declared
floor against `src.__version__` and refuses when the plugin needs a newer
core. The check sits in `install_plugin` because `_reinstall_with_rollback`
calls it, so a refused *update* restores the version the user already had.
Refusal requires evidence: an undeclared floor, an unparseable version on
either side, or an untrustworthy core version all allow the install.
- **A failed install no longer destroys the plugin it replaced.**
`install_plugin` previously deleted the existing plugin directory before
downloading, so any later failure — a dropped connection, a malformed
manifest, or the new compatibility refusal — left the user with nothing. The
existing copy is now set aside and restored if the install fails, matching
the protection `_reinstall_with_rollback` already gave the update path.
- `web_interface.__version__` re-exports `src.__version__` instead of carrying
its own hardcoded `"3.0.0"`, which had drifted two majors from the core.
- **Live games are no longer dropped when the feed omits a game clock.** - **Live games are no longer dropped when the feed omits a game clock.**
`SportsLive._is_game_really_over` previously (in the baseball and UFC `SportsLive._is_game_really_over` previously (in the baseball and UFC
plugin lineages) coerced a missing or non-string clock to the literal plugin lineages) coerced a missing or non-string clock to the literal
@@ -130,12 +86,6 @@ would lock those users out of the plugin store entirely.
there means kickoff rather than expiry. there means kickoff rather than expiry.
### Fixed ### Fixed
- **Plugin updates could hang the web request thread.** The per-plugin reinstall
locks were non-reentrant, and `_reinstall_with_rollback` holds one across its
call to `install_plugin` — which now takes the same lock to protect the
set-aside/restore above. That nesting deadlocked
`update_plugin → _reinstall_with_rollback → install_plugin`, the standard
path for every monorepo plugin update. The locks are now `RLock`s.
- `FontManager` resolves `assets/fonts` against the core install root instead - `FontManager` resolves `assets/fonts` against the core install root instead
of the process working directory, so font loading works when the process of the process working directory, so font loading works when the process
starts elsewhere (e.g. the plugin safety harness on CI). starts elsewhere (e.g. the plugin safety harness on CI).
+1 -32
View File
@@ -395,37 +395,6 @@ class PluginManager:
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=e) self.state_manager.set_state(plugin_id, PluginState.ERROR, error=e)
return False return False
#: Config keys the **core** reads out of a plugin's own config block. The
#: plugin never declares them, so a schema with
#: ``"additionalProperties": false`` — 37 of the 42 published ones — reports
#: them as violations and the plugin gets flagged degraded in the web UI for
#: using a documented core feature.
#:
#: Listed explicitly rather than matched on a ``vegas_`` prefix, because
#: ``vegas_mode`` is the opposite case: plugins *do* declare that one, and a
#: prefix rule would silently stop validating it.
#:
#: Read by: ``vegas_mode/plugin_adapter.py`` (``vegas_width_pct``,
#: ``vegas_overflow``) and ``base_plugin.py`` (``vegas_max_width_screens``).
CORE_OWNED_CONFIG_KEYS = frozenset({
'vegas_width_pct',
'vegas_overflow',
'vegas_max_width_screens',
})
def _strip_core_owned_keys(self, config: Dict[str, Any]) -> Dict[str, Any]:
"""A shallow copy of ``config`` without the core's own tuning keys.
Only the top level is touched, and only when such a key is present, so
the common case allocates nothing extra.
"""
if not isinstance(config, dict):
return config
if not self.CORE_OWNED_CONFIG_KEYS.intersection(config):
return config
return {k: v for k, v in config.items()
if k not in self.CORE_OWNED_CONFIG_KEYS}
def _validate_config_schema_soft(self, plugin_id: str, config: Dict[str, Any]) -> None: def _validate_config_schema_soft(self, plugin_id: str, config: Dict[str, Any]) -> None:
"""Validate a plugin's config against its JSON schema — warn/degrade only. """Validate a plugin's config against its JSON schema — warn/degrade only.
@@ -450,7 +419,7 @@ class PluginManager:
try: try:
is_valid, errors = self.schema_manager.validate_config_against_schema( is_valid, errors = self.schema_manager.validate_config_against_schema(
self._strip_core_owned_keys(config), schema, plugin_id config, schema, plugin_id
) )
except Exception as e: # pragma: no cover - defensive except Exception as e: # pragma: no cover - defensive
# Validation machinery itself failed — do not penalise the plugin. # Validation machinery itself failed — do not penalise the plugin.
-98
View File
@@ -1,98 +0,0 @@
"""The core's own tuning keys must not make a plugin look broken.
`vegas_width_pct`, `vegas_overflow` and `vegas_max_width_screens` are read by
the *core* out of each plugin's config block — `vegas_mode/plugin_adapter.py`
and `base_plugin.py`. No plugin declares them, and 37 of the 42 published
config schemas set `"additionalProperties": false`, so schema validation
reported them as violations.
That is not just log noise: `_validate_config_schema_soft` sets `degraded` in
the health tracker, which the web UI surfaces. Measured on a real device, **9
of 27 installed plugins** were flagged degraded purely for using a documented
core feature including `baseball-scoreboard` and `f1-scoreboard`.
The fix strips those keys before validating. It deliberately does *not* match
on a `vegas_` prefix: `vegas_mode` is plugin-owned and declared in schemas, and
a prefix rule would silently stop validating it.
"""
from unittest.mock import MagicMock
import pytest
from src.plugin_system.plugin_manager import PluginManager
STRICT_SCHEMA = {
"type": "object",
"additionalProperties": False,
"properties": {
"enabled": {"type": "boolean"},
"vegas_mode": {"type": "string"}, # plugin-owned, must stay validated
},
}
@pytest.fixture
def manager():
mgr = PluginManager.__new__(PluginManager) # skip the heavy constructor
mgr.logger = MagicMock()
mgr.schema_manager = MagicMock()
mgr._set_degraded_safe = MagicMock()
return mgr
class TestStripCoreOwnedKeys:
def test_removes_every_core_owned_key(self, manager):
cfg = {"enabled": True, "vegas_width_pct": 50,
"vegas_overflow": "wrap", "vegas_max_width_screens": 2}
assert manager._strip_core_owned_keys(cfg) == {"enabled": True}
def test_leaves_plugin_owned_vegas_mode_alone(self, manager):
"""A prefix rule would have eaten this one."""
cfg = {"enabled": True, "vegas_mode": "scroll"}
assert manager._strip_core_owned_keys(cfg) == cfg
def test_returns_the_same_object_when_nothing_to_strip(self, manager):
cfg = {"enabled": True}
assert manager._strip_core_owned_keys(cfg) is cfg
def test_does_not_mutate_the_caller_config(self, manager):
cfg = {"enabled": True, "vegas_width_pct": 50}
manager._strip_core_owned_keys(cfg)
assert "vegas_width_pct" in cfg, "the live plugin config was mutated"
def test_tolerates_a_non_dict(self, manager):
assert manager._strip_core_owned_keys(None) is None
class TestSoftValidation:
def _validate_with(self, manager, config, valid=True, errors=()):
manager.schema_manager.load_schema.return_value = STRICT_SCHEMA
manager.schema_manager.validate_config_against_schema.return_value = (
valid, list(errors))
manager._validate_config_schema_soft("baseball-scoreboard", config)
return manager.schema_manager.validate_config_against_schema.call_args
def test_core_keys_never_reach_the_validator(self, manager):
"""The regression: these keys reaching a strict schema is what flagged
9 of 27 plugins degraded."""
args = self._validate_with(
manager, {"enabled": True, "vegas_width_pct": 50})
validated = args[0][0]
assert "vegas_width_pct" not in validated
assert validated == {"enabled": True}
def test_plugin_owned_keys_still_reach_the_validator(self, manager):
args = self._validate_with(
manager, {"enabled": True, "vegas_mode": "scroll"})
assert args[0][0]["vegas_mode"] == "scroll"
def test_a_genuine_violation_is_still_reported(self, manager):
"""Stripping core keys must not turn the check into a no-op."""
self._validate_with(
manager, {"enabled": True, "typo_key": 1},
valid=False, errors=["Field root: 'typo_key' was unexpected"])
manager._set_degraded_safe.assert_called()
reason = manager._set_degraded_safe.call_args[0][1]
assert reason and "typo_key" in reason