Compare commits

..
Author SHA1 Message Date
ChuckBuildsandClaude Opus 5 0339051f0c fix(plugins): report updates that completed, not ones that were queued
run_scheduled_updates_with_changes() snapshotted plugin_last_update,
called run_scheduled_updates(), and diffed the two to answer "whose data
just changed".

But run_scheduled_updates() only enqueues. The work runs on the update
worker and stamps plugin_last_update there, after this method has already
returned, so the two snapshots were always identical and the result was
always an empty list. The only path that ever worked was the synchronous
kill-switch, where update() runs inline.

Vegas is the caller. That empty list is what feeds mark_plugin_updated(),
which drops the cached content for a plugin whose data moved -- so a
segment kept scrolling whatever it was first built from. It is the
failure the coordinator's own comments describe: last night's live game
still drawn as live the next morning. On a live rig: zero update ticks in
twenty minutes, with weather, stocks and news all updating on schedule.

The worker now records each completed update in a ledger and the call
drains it, reporting what has finished since the previous poll rather
than what this call enqueued. That costs one tick of latency -- Vegas
polls every ~4s -- and is correct whichever side of the queue the work
lands on. Failure paths are excluded: they stamp the timestamp too, to
space out retries, but no fresh data exists.

Verified on the rig it was found on: 0 update ticks before, 208 in
twenty-five minutes after, naming real plugins.

The behavioural tests here would pass with both production call sites
deleted, which mutation testing caught -- they drive the ledger directly.
So there is also a structural test asserting the invariant at the source:
wherever a successful update stamps plugin_last_update, it must record
the completion. Writing it immediately caught that _record_update_failure
stamps the same field and must not be included.

Mutation-checked: removing either call site, removing both, and dropping
the drain's clear are all caught.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5
2026-08-14 14:46:56 -04:00
3 changed files with 256 additions and 134 deletions
+33 -116
View File
@@ -212,9 +212,9 @@ one of them is safe by construction and the other is not.
| **B1** | Promote the nine universal methods; convert `sports.py` → package | ✅ | Characterization suite green; no behavior change intended |
| **B2** | `CelebrationMixin` + rotation strategies as opt-in capabilities | ✅ | Non-adopters have zero new code in their MRO; strategies checked against verbatim plugin transcriptions |
| **B3** | Upstream the scroll **orchestration** layer as `src/common/sports_scroll.py`, reading `global_config['target_fps']` natively | ✅ | Content building stays per-sport |
| **B4** | Ship 3.2.0 *and* make version reporting trustworthy | ✅ | Released 2026-08-03; tag, release and `src.__version__` agree; compatibility gate merged (#428, #431, #433) |
| **B5** | Adoption — guarded core imports, all eight. **Bundled copies stay.** | ✅ | All eight adopted; harness byte-identical; see the B5 retrospective below — four shipped broken and were repaired in plugins #251 |
| **B6** | Sunset — delete the bundled copies | **blocked, deliberately** | 3.2.0 *in users' hands*. Released 2026-08-03; there is no adoption data yet. See "B6 — the decision as of 2026-08-05" |
| **B4** | Ship 3.2.0 *and* make version reporting trustworthy | **next** | Tag, release, and `src.__version__` agree; compatibility gate merged |
| **B5** | Adoption — guarded core imports: three pilots, then the remaining six. **Bundled copies stay.** | after B4 | Per plugin: harness + goldens byte-identical, then a device soak |
| **B6** | Sunset — delete the bundled copies | **blocked** | B4's gate shipped *and* in users' hands (see below) |
### B4 — what "ship 3.2.0" actually requires
@@ -265,13 +265,10 @@ migrate there.)
### B5 — adoption is safe by construction
A plugin adopting core imports keeps its bundled copy and reaches it through the
guarded import (see the Upgradability table above). On a core that doesn't ship
the module the plugin falls back and behaves exactly as it does today. That
fallback compatibility is safe by construction. On a core that *does* ship the
module, correctness is not automatic — object-level and scroll-mode validation
(building both classes and comparing, per the retrospective below) is required
to prove full behavior. There is no version of this step that breaks a user *on
an old core*, which is why it does not wait for B6's gate.
guarded import (see the Upgradability table above). On a core that ships the
module the plugin uses core code; on one that doesn't it falls back and behaves
exactly as it does today. There is no version of this step that breaks a user,
which is why it does not wait for B6's gate.
The hockey scroll-display pilot is **already validated**: adopted against a core
carrying 3.2.0, `scroll_display.py` went from 691 to 289 lines and all 16 harness
@@ -324,115 +321,35 @@ gate rather than trusting the failure to be noticed.
The same suite should exercise the install/update gate, since it is the other
half of the guarantee.
### B6 — the decision as of 2026-08-05
**Do not run B6 yet. Do not abandon it either.** The blocker is no longer
technical; it is calendar time, and it is the one thing here that cannot be
worked around by writing more code.
**Why not yet.** 3.2.0 was published **2026-08-03**. Its predecessor 3.1.0 ran
for nine months. B6's entire safety argument is "cores without
`src.common.sports_scroll` are gone", and two days after release that is not
close to true. There are no release assets to count and no install telemetry, so
we cannot demonstrate otherwise — and that absence of evidence *is* the answer.
Executing B6 now would strand essentially the whole user base on their current
plugin versions.
**What is already done and waiting.** The hard part is built and tested. The
install gate refuses a plugin whose floor exceeds the core's version, and
refuses one whose floor is above 2.0.0 when the core reports an untrustworthy
version — so a v3.1.0-release user (who reports `1.0.0`) keeps a working plugin
instead of receiving one that cannot load. Every adopted plugin has a
`test_core_fallback.py` covering both paths.
**What would unblock it.** Evidence of 3.2.0 uptake — a few months of it being
the default download, or store-side install data if that is ever added. Revisit
then, not on a schedule.
**When it happens, remember:** four plugins declare their floor top-level, where
editing `versions[0]` is a silent no-op, and the floor has three live spellings
(`min_ledmatrix_version`, `requires.min_ledmatrix_version`,
`versions[].ledmatrix_min_version`, plus deprecated `ledmatrix_min`). See
`src/plugin_system/compatibility.py:declared_min_version` for the resolution
order any floor-raising tool must reproduce.
### B5 retrospective — what the adoption actually cost
Recorded because it is the evidence behind the two decisions above, and because
"the adoption went fine" is not what happened.
**Four of the eight shipped with scroll mode broken** on a 3.2.0 core, and were
repaired in plugins-repo #251. The restructure lifted the content methods
verbatim but left the state they read off `self` behind: separator-icon
constants (hockey, basketball, lacrosse) and the game-renderer cache (afl).
hockey/basketball/lacrosse could not construct the scroll display at all; afl
raised inside `prepare_scroll_content`, which the core base *catches*, so its
only symptom was scroll mode silently drawing nothing.
Three things are worth carrying forward:
- **The bundled fallback did not protect anyone from this.** The break was on
the modern path, which the fallback never touches. Carrying the second copy
bought nothing against the actual defect while creating the divergence that
produced it. That is an argument *for* B6, not against it.
- **Every gate was green.** The safety harness renders the scoreboard screens,
not scroll mode; `test_core_fallback.py` checked that methods existed and that
their *globals* resolved, and `self.NHL_SEPARATOR_ICON` is an attribute read,
invisible to an AST scan for `Name` loads. The fix was to stop reasoning about
source and **build the object**: construct both classes on both paths, compare
the separator icons they end up with, and assert the adopted class ends up
with every instance attribute the bundled one sets.
- **Test what the change touches, not what is convenient to render.** Scroll
mode had no coverage because the harness could not reach it. A comparison
harness that renders the same games through both paths and diffs the pixels
needs no per-sport knowledge of the right answer, only that adopting core code
did not change it.
**The ledger.** Before adoption, eight duplicated copies totalled 5,685 lines.
After adoption plus the frozen legacy copies it was 10,610; removing the dead
inline duplication (plugins #252) brought it to roughly 8,620. B6 would take it
to about 3,300 including the shared core module — some 2,400 fewer than before
this project started. **Until B6 runs, the adoption is net negative on disk**,
and its one delivered user-visible gain is that adopted plugins honour the
global `target_fps` instead of hardcoding ~100 FPS.
### Decision: stop adopting further modules until B6 closes
`data_sources.py` (9 copies), `game_renderer.py` (8) and `base_odds_manager.py`
are the obvious next candidates. **Do not adopt them yet.** Each adoption adds
carrying cost — a second copy to keep in step — against a payoff that is
contingent on B6, and B6 is gated on an installed base we cannot currently
measure. Consolidate what is already committed; revisit when B6 does.
## What's next
Steps 15 of the original plan are **done**: 3.2.0 is tagged and published with
a version number CI now asserts (#428), the compatibility gate is in
`install_plugin` and reads `compatible_versions` as well as the floor
(#431, #433), the newest manifest entry is required to use `ledmatrix_min_version`
(plugins #244), and all eight plugins have adopted the scroll orchestration
(plugins #245#249, repaired in #251, tidied in #252).
In order. Each step is independently useful and independently revertible.
What actually remains, smallest first:
1. **Nothing on the critical path.** B6 is the only remaining phase and it is
waiting on calendar time, not on work. Resist the urge to fill the gap by
adopting more modules — see the decision above.
2. **The stale plugin-test tranche** — 5 failures across baseball, hockey and
basketball, all pre-existing API drift in the plugins' own older tests
(`plugin.initialized`, `CacheManager(config_manager=...)`, a bare
`cache_manager` import, `MockLogger.setLevel`, `BasketballPluginManager`).
None are scroll-related. They make the suite noisy, which is how a real
failure gets ignored.
3. **Soak the remaining adoptions on hardware.** Only baseball has been watched
through a live game, and hockey has been loaded on devpi. The other six are
proven by harness, unit tests and pixel comparison — not by a live match.
Out-of-season sports cannot be soaked until their season starts.
4. **`CLAUDE.md` in the plugins repo says four panel sizes; the harness renders
eight.** A one-line doc fix, and the discrepancy has already produced one
false review finding.
5. **Then, when the evidence supports it, B6** — with the four-case
compatibility regression test above in CI first.
1. **Tag and publish v3.2.0.** The code is already on `main` (`21825cbf`).
Nothing else blocks this, and it is what makes `ledmatrix_min_version:
"3.2.0"` refer to something real.
2. **Make the version number honest.** Have the release process assert that the
tag, the GitHub release, and `src.__version__` agree — a check in CI is
cheaper than the confusion of the last two releases. Then revisit the
`< 2.0.0` skip in `_warn_if_incompatible`, which currently silences the
warning for the users who most need it.
3. **Add the compatibility gate** to `StoreManager.install_plugin` and
`.update_plugin`: refuse a plugin whose declared floor exceeds
`src.__version__`, and surface the reason in the store UI rather than only
the log. This is the single change that turns the floor from documentation
into a guarantee, and B6 depends on it.
4. **Migrate the manifests** to `ledmatrix_min_version`, and reconcile them with
`compatible_versions` (see above — that field is the required, canonical one,
and the gate does not read it yet). Currently 28 plugins spell the floor both
ways across their `versions[]` entries, 12 use only the old spelling, and 2
only the new. Scope the sweep to the nine sports plugins if a 42-plugin
version-bump wave isn't worth it — but the `compatible_versions` half has to
cover every manifest the gate can refuse, or define explicit legacy handling,
before the gate is allowed to block anything.
5. **Run B5 adoption** — hockey, soccer, football, then the remaining six.
Bundled copies stay. Byte-identical harness output per plugin, then a soak.
6. **Only then plan B6**, with the compatibility regression test described above
in CI first.
## How to keep this project healthy
+41 -18
View File
@@ -116,6 +116,14 @@ class PluginManager:
self._plugin_locks: Dict[str, threading.Lock] = {}
self._plugin_locks_guard = threading.Lock()
self._update_worker: Optional[threading.Thread] = None
# Plugin ids whose update() has finished since the last time anyone
# asked. Updates are dispatched to a worker thread, so a caller that
# wants to know "whose data just changed" cannot learn it by diffing
# plugin_last_update around run_scheduled_updates() -- that call only
# enqueues, and the timestamp is stamped later, on the worker. See
# run_scheduled_updates_with_changes().
self._completed_updates: set = set()
self._completed_updates_lock = threading.Lock()
self._synchronous_updates = False
if self.config_manager is not None:
try:
@@ -940,6 +948,7 @@ class PluginManager:
if success:
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = scheduled_time
self._note_update_completed(plugin_id)
self.state_manager.record_update(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
if self.health_tracker:
@@ -1006,28 +1015,41 @@ class PluginManager:
def run_scheduled_updates_with_changes(self, current_time: Optional[float] = None) -> List[str]:
"""
Like run_scheduled_updates(), but also returns the plugin_ids whose
plugin_last_update timestamp actually advanced during this call.
Like run_scheduled_updates(), but also reports which plugins have
fresh data -- the ids whose update() has finished since the last
call, not necessarily the ones enqueued by this one.
The before/after snapshots and the update pass itself are each
individually lock-protected against concurrent plugin_last_update
mutation (Vegas mode calls this from its own background
update-tick thread, racing the main render loop's plugin updates),
so callers get an atomic "who got fresh data" answer without
reaching into plugin_last_update themselves. The lock is not held
across the update pass so slow/blocking plugin update() calls don't
serialize against other plugin_last_update readers.
That distinction is the whole point. This used to snapshot
plugin_last_update, call run_scheduled_updates(), and diff. But
run_scheduled_updates() only *enqueues*: the work runs on the
update worker and the timestamp is stamped there, after this method
has already returned. The two snapshots were therefore always
identical and the result was always empty, so Vegas never learned
that any plugin's data had changed and kept scrolling whatever a
segment was first built from -- last night's live game still drawn
as live the next morning. The only path that ever worked was the
synchronous kill-switch, where update() runs inline.
Reporting completions instead of enqueues costs a poll's worth of
latency (the Vegas tick runs every ~4s) and is correct regardless of
which side of the queue the work lands on.
"""
with self._plugin_last_update_lock:
old_times = dict(self.plugin_last_update)
self.run_scheduled_updates(current_time)
return self.drain_completed_updates()
with self._plugin_last_update_lock:
return [
plugin_id for plugin_id, new_time in self.plugin_last_update.items()
if new_time > old_times.get(plugin_id, 0.0)
]
def _note_update_completed(self, plugin_id: str) -> None:
"""Record that a plugin's update() finished, for the next poll."""
with self._completed_updates_lock:
self._completed_updates.add(plugin_id)
def drain_completed_updates(self) -> List[str]:
"""Return and clear the plugin ids whose update() has since finished."""
with self._completed_updates_lock:
if not self._completed_updates:
return []
done = sorted(self._completed_updates)
self._completed_updates.clear()
return done
def update_all_plugins(self) -> None:
"""
@@ -1053,6 +1075,7 @@ class PluginManager:
if success:
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = time.time()
self._note_update_completed(plugin_id)
self.state_manager.record_update(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
else:
+182
View File
@@ -0,0 +1,182 @@
#!/usr/bin/env python3
"""
Tests that "which plugins have fresh data" survives the async update worker.
Regression under test: run_scheduled_updates_with_changes() snapshotted
plugin_last_update, called run_scheduled_updates(), and diffed the two. But
run_scheduled_updates() only *enqueues* -- the work runs on the update worker
and stamps the timestamp there, after the method has already returned. The
snapshots were therefore always identical and the result always empty.
Vegas depends on that result: it is what calls mark_plugin_updated(), which
drops the cached content for a plugin whose data changed. With it always
empty, a segment kept scrolling whatever it was first built from -- the
"last night's live game still drawn as live the next morning" failure the
coordinator comments describe. Observed on a live rig: zero update ticks in
twenty minutes, with weather, stocks and news all updating.
Run: python -m pytest test/test_update_change_reporting.py -v
"""
import ast
import inspect
import sys
import threading
import unittest
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.plugin_system.plugin_manager import PluginManager # noqa: E402
def _manager():
"""A PluginManager with only the update-reporting state initialised."""
manager = PluginManager.__new__(PluginManager)
manager._completed_updates = set()
manager._completed_updates_lock = threading.Lock()
return manager
class DrainCompletedUpdates(unittest.TestCase):
def setUp(self):
self.manager = _manager()
def test_nothing_completed_reports_nothing(self):
self.assertEqual(self.manager.drain_completed_updates(), [])
def test_a_completed_update_is_reported(self):
self.manager._note_update_completed("news")
self.assertEqual(self.manager.drain_completed_updates(), ["news"])
def test_draining_clears_so_the_next_poll_is_empty(self):
self.manager._note_update_completed("news")
self.manager.drain_completed_updates()
self.assertEqual(
self.manager.drain_completed_updates(), [],
"a plugin must be reported once per update, not on every poll, "
"or Vegas would drop its cached content every few seconds")
def test_repeated_completions_between_polls_collapse(self):
for _ in range(5):
self.manager._note_update_completed("weather")
self.assertEqual(self.manager.drain_completed_updates(), ["weather"])
def test_multiple_plugins_are_all_reported(self):
for plugin_id in ("news", "weather", "ledmatrix-stocks"):
self.manager._note_update_completed(plugin_id)
self.assertEqual(self.manager.drain_completed_updates(),
["ledmatrix-stocks", "news", "weather"])
class CompletionReportingIsAsyncSafe(unittest.TestCase):
"""The point of the change: completion may land after the call returns."""
def setUp(self):
self.manager = _manager()
def test_an_update_completing_after_the_call_is_still_reported(self):
"""The exact shape of the bug.
The enqueueing call sees nothing, because the worker has not run yet.
The next poll must report it -- under the old diff it was lost, since
the second snapshot was taken before the worker ever stamped.
"""
first = self.manager.drain_completed_updates()
self.assertEqual(first, [], "nothing has finished yet")
# The worker finishes some time later, on its own thread.
worker = threading.Thread(
target=self.manager._note_update_completed, args=("news",))
worker.start()
worker.join()
self.assertEqual(
self.manager.drain_completed_updates(), ["news"],
"an update that finishes between polls must still be reported")
def test_concurrent_completions_are_not_lost(self):
ids = ["plugin-%02d" % i for i in range(40)]
threads = [threading.Thread(target=self.manager._note_update_completed,
args=(pid,)) for pid in ids]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
self.assertEqual(self.manager.drain_completed_updates(), sorted(ids))
def test_a_completion_during_a_drain_is_not_swallowed(self):
"""A drain must not clear an entry it did not report."""
self.manager._note_update_completed("news")
reported = self.manager.drain_completed_updates()
# ...worker finishes another one immediately afterwards
self.manager._note_update_completed("weather")
self.assertEqual(reported, ["news"])
self.assertEqual(self.manager.drain_completed_updates(), ["weather"])
class EveryStampRecordsACompletion(unittest.TestCase):
"""The ledger is only correct if the production paths actually fill it.
Asserting on the mechanics alone passes even when nothing calls
_note_update_completed -- verified by deleting the call sites, which the
behavioural tests above did not notice. This checks the invariant at the
source: wherever a successful update stamps plugin_last_update, it must
also record the completion, or Vegas silently stops being told.
"""
def test_success_paths_record_the_completion(self):
import src.plugin_system.plugin_manager as pm
tree = ast.parse(inspect.getsource(pm))
stamps = []
for node in ast.walk(tree):
if not isinstance(node, ast.With):
continue
# `with self._plugin_last_update_lock:` blocks that stamp a real
# time on success. Two stamps are deliberately excluded: the 0.0
# written at registration, and the failure path, which backs the
# timestamp off to space out retries -- neither means fresh data.
assigns_time = any(
isinstance(stmt, ast.Assign)
and any(isinstance(t, ast.Subscript)
and getattr(t.value, "attr", None) == "plugin_last_update"
for t in stmt.targets)
and not (isinstance(stmt.value, ast.Constant)
and stmt.value.value == 0.0)
and "failure" not in ast.dump(stmt.value)
for stmt in node.body
)
if assigns_time:
stamps.append(node)
self.assertGreaterEqual(
len(stamps), 2,
"expected the worker and inline success paths to stamp the time; "
"if this drops, the search below is looking at the wrong thing")
for stamp in stamps:
enclosing = self._enclosing_function(tree, stamp)
calls = [n for n in ast.walk(enclosing)
if isinstance(n, ast.Call)
and getattr(n.func, "attr", None) == "_note_update_completed"]
self.assertTrue(
calls,
"%s stamps plugin_last_update on success but never calls "
"_note_update_completed, so a plugin's fresh data would never "
"be reported and Vegas would keep its stale cached content"
% enclosing.name)
@staticmethod
def _enclosing_function(tree, target):
best = None
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
if node.lineno <= target.lineno <= (node.end_lineno or node.lineno):
if best is None or node.lineno > best.lineno:
best = node
return best
if __name__ == "__main__":
unittest.main(verbosity=2)