3 Commits

Include renames

Author	SHA1	Message	Date
ChuckandClaude Opus 5.5	d18e4d3c9d	fix(plugins): sub-package reload, symlinked dev plugins, BaseException in update(), config callbacks outside the lock (#741) ... * fix(plugins): drop a plugin's package modules when it unloads A plugin that keeps helpers in a package (providers/feed.py, imported as `from providers.feed import ...`) leaves dotted entries in sys.modules. PluginLoader only tracked bare names: `providers` was namespaced and dropped on unload, `providers.feed` stayed. A reload after a store update imported a fresh `providers`, then got the old `feed` back from the module cache, so the new manager.py ran against the old helpers until the display restarted. A load that failed part-way left them behind the same way. Elections (providers/), flights (enrichment/) and olympics (data/, renderers/) ship packages. The loader now records the dotted modules whose file (or, for a namespace package, every __path__ entry) lies inside the plugin directory. They keep their names while the plugin runs, as before, and unregister_plugin_modules() drops them, only while sys.modules still holds that plugin's module. The failed-load cleanup in load_module() drops them too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(plugins): remove a symlinked dev plugin as a link PluginStoreManager._safe_remove_directory, behind uninstall and behind discarding the set-aside copy after an install or update, handed a symlinked dev plugin (scripts/dev/dev_plugin_setup.sh) to shutil.rmtree, which refuses a symlink. The chmod fallback then walked through the link and set every directory and file in the linked checkout to 0700, and the sudo stage refused the resolved path as outside the plugins directory. The removal failed, the link stayed, and the developer's checkout lost its group/other permissions. A dangling link read as already removed, because exists() follows it, and was left behind. A symlink is now unlinked before any other stage runs, and before the exists() check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(plugins): load a dev plugin linked in under a different name contained_plugin_dir(), the containment check before a plugin's dependencies are installed, resolved the plugin directory and looked for the resolved folder's name among the plugins directory's entries. A dev plugin symlinked in under its id by a name its checkout does not share -- `dev_plugin_setup.sh link-github foo <url>` clones ledmatrix-foo, the repository naming convention, and links it as plugins/foo -- has no such entry, so install_dependencies() returned False and the load failed with "Dependency installation failed", even with no requirements.txt. When the path sits directly in the plugins directory, the entry it names (the link) is looked up first; anything else is resolved and matched by name as before. The answer is still always rebuilt from a name os.scandir() returned for the plugins directory, so a path outside it is still refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(plugins): release a plugin whose update() raises a BaseException On the async update worker, the wrapped update() finished its bookkeeping (_finish: release the plugin lock, drop the pending slot, state back to ENABLED) only for an Exception. asyncio.CancelledError and SystemExit derive from BaseException, so one raised from update() skipped _finish: the plugin kept its lock and stayed RUNNING for the life of the process, never rescheduled, with every display() skipped as busy. PluginExecutor caught only Exception as well, so its thread died with the call never marked complete and an immediate failure was logged and recorded as a timeout. _target_update now runs _finish for any BaseException and re-raises it, and the executor's thread stores it like any other exception, so it is reported as the operation's failure (PluginError) on both the async and the synchronous path. _finish and _record_update_failure take a BaseException. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(config): notify config subscribers outside the service lock ConfigService._load_config ran every subscriber while holding _lock. The display's per-plugin subscriber calls PluginManager.apply_config_change, which waits up to PLUGIN_LOCK_TIMEOUT (5 s) for a plugin busy in update(). A save that enables or disables a plugin also flags a reconcile, which the render thread runs: its get_config(), and the unsubscribe() of a plugin it disables, both take _lock, so the panel froze behind every slow callback, up to 5 s per busy plugin. The config is now swapped under _lock and the subscribers are called after it is released, from a copy of the subscriber lists. A separate _notify_lock is held across a whole reload (read, swap, notify), so one reload's notifications still finish before the next one's start. Each callback is checked against the live lists just before it runs, and unsubscribe() waits only for a call of that same callback already in progress (unless it is that callback's own thread), so a callback it removed is not running and will not run once it returns, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-03 22:18:42 -04:00
ChuckandClaude Opus 5.5	c00bf5e8e6	fix(plugin-system): unload/update race, failed-load cleanup, limits validation, schema lookup, install rollback (#653) ... * fix(plugin-system): unload/update race, failed-load module cleanup, limits validation, schema lookup, install rollback, op-queue dedupe - unload_plugin takes the per-plugin lock (5s bounded) before cleanup(), and an update() that finishes after its plugin was unloaded no longer sets the state back to ENABLED. - A load that fails after import drops plugin_<id> and its submodules and forgets its manager fonts, so a fixed plugin reloads new code. - Resource limits are validated as non-negative numbers: 400 at POST /plugins/limits, bad cached records ignored with one warning. Route docstrings note health/metrics reset and limits only change the web process's view. - SchemaManager.get_schema_path resolves each search dir via resolve_plugin_dir (manifest id, ledmatrix-<id>) before the literal paths; plugins/ still before plugin-repos/. Misses cached 30s and logged once at DEBUG. - install_from_url sets an existing copy aside and restores it if the move fails, under the per-plugin reinstall lock. - Operation queue refuses a second pending op for a plugin and trims _operations with history. - get_vegas_render_width reads display_manager.width first. - get_logger in store/schema/health/resource/saved_repositories; UTF-8 reads in store_manager and state_manager. - Docs: update_interval precedence (manifest over config) stated where users are told to set it in config. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): build the limits 400 message from the field name, not an exception CodeQL flagged str(e) flowing into the response. invalid_limit_field() returns the offending field without raising, and limits_from_dict uses it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-28 10:41:40 -04:00
3 people	0aca40cf3a	perf(plugins): run scheduled updates off the render thread (#407) ... * perf(plugins): run scheduled updates off the render thread plugin update() executed inline in the render loop — execute_update's internal thread.join(timeout=30) blocked it, so one slow plugin HTTP fetch froze scrolling for the whole fetch (up to 30s; DNS-retry storms made this a regular occurrence on flaky networks). Scheduling stays on the render thread and keeps every existing gate (enabled, circuit breaker, can_execute, interval); due updates are now enqueued to a single background worker (serialized — same one-at-a-time execution as before, no thundering herd). RUNNING is set at enqueue so can_execute blocks re-entry alongside the pending-set dedup. Per-plugin locks make the old implicit update/display no-overlap guarantee explicit: the worker holds the plugin's lock through its update; the display side try-locks and, when the plugin is mid-update, holds the last frame for that iteration — reported as success so a mid-update skip never advances the rotation. Unlike before, the guarantee now also holds across the post-timeout window (previously the lingering update thread overlapped display()). Deadlock-free by construction: the worker takes one lock; display never blocks. Timeout semantics unchanged (lingering daemon thread documented). Kill switch: plugin_system.synchronous_updates: true restores the inline path. 8 new concurrency tests (non-blocking scheduler, overlap assertion under a hammering display loop, lock release on failure/timeout paths, dedup, kill switch); 4-min devpi soak clean (updates completing, rotation advancing, no stuck RUNNING states). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(plugins): fix skipped-frame health/force_change tracking, config validation, and lock-lifetime gaps in async updates Addresses PR #407 review findings: - display_controller: only clear force_change / record health success when display() actually ran this frame, not when the frame was skipped because the plugin's lock was busy (a skip must preserve a pending mode-switch force_clear). - plugin_manager: replace bool() coercion of synchronous_updates with explicit isinstance validation of plugin_system/synchronous_updates, failing safe to synchronous mode (with a logged reason) on malformed config instead of silently defaulting to async. - plugin_manager: _update_worker_loop now acquires the plugin lock before looking up its instance and re-checks under the lock, so an unloaded plugin's lifecycle state is never resurrected to ENABLED. - plugin_manager + display_controller: move lock ownership (and, for updates, RUNNING/pending lifecycle bookkeeping) into the actual update()/ display() call itself rather than the timeout-wrapped caller, so the lock stays held for the real operation's duration even after PluginExecutor's own join(timeout) elapses and a lingering daemon thread keeps running in the background. - DisplayController.cleanup() now stops the update worker before tearing down display/cache resources; stop_update_worker() logs when the join times out instead of failing silently. - test_async_plugin_updates: rewrite test_unloaded_while_queued_is_harmless to exercise the public unload_plugin() lifecycle (via a deterministic blocker) instead of deleting pm.plugins directly, and add a regression test proving the plugin lock stays held through PluginExecutor's own timeout while the real update() call is still running. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-14 08:08:51 -04:00