rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-04 06:15:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(web): read plugins through a PluginCatalog; only the display runs them (#688)

Browse Source 
The web process built its own PluginManager and loaded plugins into itself:
store installs and updates loaded or reloaded a web-side copy, and config
saves and enable/disable called on_config_change, on_enable and on_disable
on it. None of that reached the panel, and /plugins/installed reported
runtime state from those copies.

- Add PluginCatalog (src/plugin_system/plugin_catalog.py): manifests,
  directories, display modes, installed version, schema and config reads,
  with no way to run a plugin. app.py and both blueprints use it; the
  plugin_manager blueprint attribute is gone.
- Remove every lifecycle call from the web routes. Config changes already
  reach the display through ConfigService (on_config_change) and the
  enabled-set reconcile.
- Health and metrics readers move to api_v3.health_tracker /
  resource_monitor. /plugins/installed reports loaded/state/error_info as
  null (the display does not publish them) and enabled by the display's
  rule.
- Store install, update and uninstall answer restart_required when the
  running display will not pick the change up by itself
  (display_restart_required). The restart banner follows the flag via
  window.noteRestartRequired instead of the /config/main URL heuristic;
  /config/main now sends restart_required: true.
- The one remaining in-process import of plugin code (Starlark helper
  modules, oauth_flow action scripts) goes through
  _import_plugin_code_in_web_process() until a web-entry contract.
- /plugins/installed reports vegas_participation (from #682) from the
  user's setting or the manifest, with vegas_participation_source; when
  only the plugin's code decides it, null with source 'runtime', since the
  web process no longer has plugin instances to ask.
- Check & Update All keeps its restart flags when the final list refresh
  fails, and asks for a restart when an enabled plugin's first request got
  no answer and the re-sent one found it up to date.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-30 10:39:44 -04:00
committed by GitHub
co-authored by Claude Opus 5.5
parent ba6eccb489
commit 7ab6fb1aff
74 changed files with 1759 additions and 676 deletions
Download Patch File Download Diff File Expand all files Collapse all files
web_interface/app.py
+27 -19
View File
@@ -34,7 +34,7 @@ from src.common.path_safety import (
)
from werkzeug.exceptions import HTTPException
from src.exceptions import ConfigError
from src.plugin_system.plugin_manager import PluginManager
from src.plugin_system.plugin_catalog import PluginCatalog
from src.plugin_system.store_manager import PluginStoreManager
from src.plugin_system.saved_repositories import SavedRepositoriesManager
from src.plugin_system.schema_manager import SchemaManager
@@ -117,12 +117,6 @@ else:
# If relative, resolve relative to the project root
plugins_dir = project_root / plugins_dir_name
plugin_manager = PluginManager(
plugins_dir=str(plugins_dir),
config_manager=config_manager,
display_manager=None, # Not needed for web interface
cache_manager=None # Not needed for web interface
)
plugin_store_manager = PluginStoreManager(plugins_dir=str(plugins_dir))
# A core `git pull` update (or any checkout) restores built-in plugins
# committed under plugin-repos/, even ones the user uninstalled. Re-remove any
@@ -149,6 +143,18 @@ schema_manager = SchemaManager(
config_manager=config_manager
)
# The web process reads plugins as files and never runs them: no plugin module
# is imported, no plugin class instantiated, no lifecycle hook called here.
# Only the display process (src/display_controller.py) does that. Config
# saves reach the running plugins through the display's config watcher; what
# the display knows at run time (health, metrics, errors, current mode) it
# publishes to the shared cache. See docs/ARCHITECTURE.md.
plugin_catalog = PluginCatalog(
plugins_dir=plugins_dir,
config_manager=config_manager,
schema_manager=schema_manager,
)
# Initialize operation queue for plugin operations
operation_queue = PluginOperationQueue(max_history=500)
@@ -169,7 +175,7 @@ operation_history = OperationHistory(
)
# Plugin discovery is deferred until first API request that needs it
# This improves startup time - endpoints will call discover_plugins() when needed
# This improves startup time - endpoints call plugin_catalog.discover_plugins() when needed
# Register blueprints
from web_interface.blueprints.pages_v3 import pages_v3
@@ -177,13 +183,13 @@ from web_interface.blueprints.api_v3 import api_v3
# Initialize managers in blueprints
pages_v3.config_manager = config_manager
pages_v3.plugin_manager = plugin_manager
pages_v3.plugin_catalog = plugin_catalog
pages_v3.plugin_store_manager = plugin_store_manager
pages_v3.saved_repositories_manager = saved_repositories_manager
pages_v3.schema_manager = schema_manager
api_v3.config_manager = config_manager
api_v3.plugin_manager = plugin_manager
api_v3.plugin_catalog = plugin_catalog
api_v3.plugin_store_manager = plugin_store_manager
api_v3.saved_repositories_manager = saved_repositories_manager
api_v3.schema_manager = schema_manager
@@ -194,17 +200,19 @@ api_v3.operation_history = operation_history
from src.cache_manager import CacheManager
api_v3.cache_manager = CacheManager()
# Wire plugin health/metrics for the web process. The display service records
# health and execution-time metrics to the shared on-disk cache; giving the web
# process its own tracker/monitor backed by that same cache lets the health API
# routes (/api/v3/plugins/health, /plugins/metrics) read that persisted data.
# Plugin health and metrics as the display publishes them. The display service
# records health and execution-time metrics to the shared on-disk cache; a
# tracker/monitor backed by that same cache lets the health API routes
# (/api/v3/plugins/health, /plugins/metrics) read what it wrote.
# Guarded so any init failure degrades to "not available" rather than breaking
# the web server.
api_v3.health_tracker = None
api_v3.resource_monitor = None
try:
from src.plugin_system.plugin_health import PluginHealthTracker
from src.plugin_system.resource_monitor import PluginResourceMonitor
plugin_manager.health_tracker = PluginHealthTracker(api_v3.cache_manager)
plugin_manager.resource_monitor = PluginResourceMonitor(api_v3.cache_manager)
api_v3.health_tracker = PluginHealthTracker(api_v3.cache_manager)
api_v3.resource_monitor = PluginResourceMonitor(api_v3.cache_manager)
except Exception as _hm_err: # pragma: no cover - defensive startup guard
logging.getLogger(__name__).warning(
"Could not enable plugin health/metrics for web UI: %s", _hm_err
@@ -960,7 +968,7 @@ def _run_startup_reconciliation() -> None:
reconciler = StateReconciliation(
state_manager=plugin_state_manager,
config_manager=config_manager,
plugin_manager=plugin_manager,
plugin_manager=plugin_catalog,
plugins_dir=plugins_dir,
store_manager=plugin_store_manager
)
@@ -968,7 +976,7 @@ def _run_startup_reconciliation() -> None:
if result.inconsistencies_found:
_logger.info("[Reconciliation] %s", result.message)
if result.inconsistencies_fixed:
plugin_manager.discover_plugins()
plugin_catalog.discover_plugins()
if not result.reconciliation_successful:
_logger.warning(
"[Reconciliation] Finished with %d unresolved issue(s); "
@@ -1043,7 +1051,7 @@ def start_auto_update_scheduler():
config_manager=config_manager,
core_update=perform_core_update,
store_manager=plugin_store_manager,
plugin_manager=plugin_manager,
plugin_catalog=plugin_catalog,
schema_manager=schema_manager,
operation_history=operation_history,
)
web_interface/auto_update.py
+7 -4
View File
@@ -329,7 +329,7 @@ class AutoUpdater:
"""Decides when an automatic update is due and runs it."""
def __init__(self, config_manager, core_update, store_manager=None,
plugin_manager=None, schema_manager=None, operation_history=None,
plugin_catalog=None, schema_manager=None, operation_history=None,
project_root=PROJECT_ROOT, state_file=None, clock=time.time,
restart=restart_service, run=subprocess.run,
service_active=_service_active, helper_ready=helper_ready,
@@ -337,7 +337,10 @@ class AutoUpdater:
self.config_manager = config_manager
self.core_update = core_update
self.store_manager = store_manager
self.plugin_manager = plugin_manager
# The web process's PluginCatalog: rescanned after plugin updates.
# The display picks the new code up when _run_deferred_plugins
# restarts it.
self.plugin_catalog = plugin_catalog
self.schema_manager = schema_manager
self.operation_history = operation_history
self.project_root = Path(project_root)
@@ -658,9 +661,9 @@ class AutoUpdater:
for plugin_id in updated:
if self.schema_manager:
self.schema_manager.invalidate_cache(plugin_id)
if updated and self.plugin_manager:
if updated and self.plugin_catalog:
try:
self.plugin_manager.discover_plugins()
self.plugin_catalog.discover_plugins()
except Exception:
logger.debug("discover_plugins after auto-update failed", exc_info=True)
return updated, failed
web_interface/blueprints/api_v3/__init__.py
+129 -119
View File
@@ -91,8 +91,9 @@ def _scrub_git_remote_url(url: str) -> str:
pass
return url
# NOTE: the managers live on the blueprint object (app.py sets
# api_v3.config_manager, api_v3.plugin_manager, api_v3.cache_manager and
# the rest). Deliberately not mirrored as module globals: a bare
# api_v3.config_manager, api_v3.plugin_catalog, api_v3.cache_manager and
# the rest). There is no plugin manager: the web process reads plugins
# through a PluginCatalog and never runs them (docs/ARCHITECTURE.md). Deliberately not mirrored as module globals: a bare
# `config_manager` used to resolve to a None that was never assigned, which
# silently disabled the /health checks and made /display/current fall back
# to a hardcoded 128x64.
@@ -631,7 +632,7 @@ def _non_plugin_id_error(plugin_id):
status_code=400)
return None
def _discovered_plugin_manifests(plugin_id=None, rescan=False):
"""The plugin manager's manifests, discovering plugins first if needed.
"""The plugin catalog's manifests, discovering plugins first if needed.
The web process discovers plugins lazily (see app.py): nothing scans at
startup, so plugin_manifests is empty until some endpoint calls
@@ -646,18 +647,18 @@ def _discovered_plugin_manifests(plugin_id=None, rescan=False):
Otherwise the existing map is reused, so a steady stream of requests
for known plugins costs nothing.
Returns the manifest map, or {} when there is no plugin manager.
Returns the manifest map, or {} when there is no plugin catalog.
"""
pm = api_v3.plugin_manager
if pm is None:
catalog = getattr(api_v3, 'plugin_catalog', None)
if catalog is None:
return {}
manifests = getattr(pm, 'plugin_manifests', None)
manifests = getattr(catalog, 'plugin_manifests', None)
if not manifests or rescan or (plugin_id is not None and plugin_id not in manifests):
try:
pm.discover_plugins()
catalog.discover_plugins()
except Exception:
logger.warning('Plugin discovery failed', exc_info=True)
manifests = getattr(pm, 'plugin_manifests', None)
manifests = getattr(catalog, 'plugin_manifests', None)
return manifests or {}
def _do_transactional_uninstall(plugin_id, preserve_config):
"""Execute an uninstall with snapshot-based rollback.
@@ -665,12 +666,13 @@ def _do_transactional_uninstall(plugin_id, preserve_config):
Order of operations:
1. Snapshot main config + secrets (abort on unexpected errors, proceed on expected I/O errors).
2. Clean up plugin config (abort with 500 if this raises — avoids orphaned files).
3. Unload plugin from runtime if loaded (rollback + 500 if this raises).
4. Remove plugin files (rollback + 500 if this returns False or raises).
5. Finish (remove state, invalidate caches).
3. Remove plugin files (rollback + 500 if this returns False or raises).
4. Finish (remove state, invalidate caches).
Rollback restores the config snapshot and, if the plugin had been
loaded before unload, calls load_plugin to restore runtime state.
Rollback restores the config snapshot. Nothing is unloaded here: the web
process never loaded the plugin. The display unloads it when the removed
config section reaches its config watcher; plugin_catalog's
display_restart_required() covers the case where that doesn't happen.
Returns (True, None) on success or (False, error_message) on failure.
"""
@@ -692,13 +694,7 @@ def _do_transactional_uninstall(plugin_id, preserve_config):
if not preserve_config:
api_v3.config_manager.cleanup_plugin_config(plugin_id, remove_secrets=True)
# Record whether the plugin was running before we touch anything.
was_loaded = (
api_v3.plugin_manager is not None
and plugin_id in api_v3.plugin_manager.plugins
)
def _rollback(reload_plugin):
def _rollback():
if main_snapshot is not None:
try:
api_v3.config_manager.save_raw_file_content('main', main_snapshot)
@@ -709,32 +705,19 @@ def _do_transactional_uninstall(plugin_id, preserve_config):
api_v3.config_manager.save_raw_file_content('secrets', secrets_snapshot)
except Exception as restore_err:
logger.error("Failed to restore secrets snapshot for %s: %s", plugin_id, restore_err)
if reload_plugin and api_v3.plugin_manager is not None:
try:
api_v3.plugin_manager.load_plugin(plugin_id)
except Exception as reload_err:
logger.error("Failed to reload plugin %s during rollback: %s", plugin_id, reload_err)
# --- Step 3: unload ---
if was_loaded:
try:
api_v3.plugin_manager.unload_plugin(plugin_id)
except Exception as unload_err:
_rollback(reload_plugin=False) # unload failed — runtime state unchanged
return False, f"Failed to unload plugin {plugin_id}: {unload_err}"
# --- Step 4: remove files ---
# --- Step 3: remove files ---
try:
success = api_v3.plugin_store_manager.uninstall_plugin(plugin_id)
except Exception as remove_err:
_rollback(reload_plugin=was_loaded)
_rollback()
return False, f"Failed to remove plugin {plugin_id}: {remove_err}"
if not success:
_rollback(reload_plugin=was_loaded)
_rollback()
return False, f"Failed to uninstall plugin {plugin_id}"
# --- Step 5: finish ---
# --- Step 4: finish ---
if api_v3.schema_manager:
api_v3.schema_manager.invalidate_cache(plugin_id)
if api_v3.plugin_state_manager:
@@ -747,6 +730,43 @@ def _do_transactional_uninstall(plugin_id, preserve_config):
except Exception as record_err:
logger.warning("Could not record uninstall for %s: %s", plugin_id, record_err)
return True, None
def _plugin_enabled_in_config(plugin_id: str) -> bool:
"""Whether config.json enables ``plugin_id``, by the display's rule.
Read this before an operation changes the config (uninstall removes the
section). A config that cannot be read counts as enabled, so the answer
errs towards asking for a restart.
"""
try:
section = (api_v3.config_manager.load_config() or {}).get(plugin_id)
except Exception:
logger.debug("Could not read config for %s", plugin_id, exc_info=True)
return True
return isinstance(section, dict) and bool(section.get('enabled', False))
_RESTART_MESSAGES = {
'install': 'Plugin installed — restart the display to start it',
'update': 'Plugin updated — restart the display to run the new version',
'uninstall': 'Plugin uninstalled — restart the display to stop it',
}
def _store_restart_fields(action: str, plugin_enabled: bool, **kwargs) -> Dict[str, Any]:
"""``restart_required`` (and the banner's wording) for a store response.
The rules are ``display_restart_required``'s: whether the display picks
the change up by itself or keeps running what it has until a restart.
The UI shows its restart banner when ``restart_required`` is true.
"""
from src.plugin_system.plugin_catalog import display_restart_required
required = display_restart_required(action, plugin_enabled, **kwargs)
fields: Dict[str, Any] = {'restart_required': required}
if required:
fields['restart_message'] = _RESTART_MESSAGES[action]
return fields
def deep_merge(base_dict, update_dict):
"""
Deep merge update_dict into base_dict.
@@ -1346,24 +1366,6 @@ def _enhance_schema_with_core_properties(schema):
return with_core_plugin_properties(schema)
def _prepared_plugin_config(plugin_id, raw_config):
"""A plugin's config section as the plugin runs with it, for on_config_change.
Loading a plugin reads legacy booleans as objects and fills in schema
defaults (PluginManager.prepare_plugin_config); a save's notification must
hand over the same shape. Falls back to the raw section.
"""
prepare = getattr(api_v3.plugin_manager, 'prepare_plugin_config', None)
if callable(prepare):
try:
prepared = prepare(plugin_id, raw_config)
if isinstance(prepared, dict):
return prepared
except Exception:
logger.debug("Could not prepare config for %s", plugin_id, exc_info=True)
return raw_config
def _filter_config_by_schema(config, schema, prefix=''):
"""
Filter config to only include fields defined in the schema.
@@ -1423,15 +1425,15 @@ _CALENDAR_LIST_MAX_PAGES = 10
def _plugin_directory(plugin_id: str) -> Optional[Path]:
"""An installed plugin's directory, or None when it has none on disk.
Only the plugin manager is asked, so no plugin manager means None. There
is no fallback to the legacy plugins/ directory: the loader never scans
it, so a plugin found only there is one that never runs.
Only the plugin catalog is asked, so no catalog means None. There is no
fallback to the legacy plugins/ directory: the loader never scans it, so
a plugin found only there is one that never runs.
"""
# getattr: the blueprint only has plugin_manager once the app has set it.
manager = getattr(api_v3, 'plugin_manager', None)
if not manager:
# getattr: the blueprint only has plugin_catalog once the app has set it.
catalog = getattr(api_v3, 'plugin_catalog', None)
if not catalog:
return None
plugin_dir = manager.get_plugin_directory(plugin_id)
plugin_dir = catalog.get_plugin_directory(plugin_id)
if not plugin_dir or not Path(plugin_dir).exists():
return None
return Path(plugin_dir)
@@ -1533,10 +1535,22 @@ _STARLARK_MANIFEST_FILE = _STARLARK_APPS_DIR / 'manifest.json'
# A dedicated, never-replaced file to flock -- see _starlark_manifest_lock.
_STARLARK_MANIFEST_LOCK_FILE = _STARLARK_APPS_DIR / 'manifest.json.lock'
def _get_starlark_plugin() -> Optional[Any]:
"""Get the starlark-apps plugin instance, or None."""
if not api_v3.plugin_manager:
return None
return api_v3.plugin_manager.get_plugin('starlark-apps')
"""The starlark-apps plugin instance in this process: always None.
The web process runs no plugin code (see PluginCatalog), so every
Starlark route takes its standalone path -- starlark-apps/manifest.json
and each app's files on disk, rendered through Pixlet directly -- and the
display's own starlark-apps plugin reads what they write. Before, this
returned a web-side copy only in the rare session that had just
installed or updated starlark-apps from the store, and that copy's
frames and state never reached the panel.
This is the one seam where a Starlark route would reach a plugin
instance. The instance branches behind it stay until the plugin
web-entry contract (docs/ARCHITECTURE.md) gives plugins an explicit way
to serve web requests; the tests drive them through this function.
"""
return None
def _find_pixlet_binary(explicit_path: Optional[str] = None) -> Optional[str]:
"""Find pixlet binary: explicit path → bundled binary → system PATH."""
import platform
@@ -1647,70 +1661,66 @@ def _starlark_github_token() -> Optional[str]:
except Exception:
logger.warning("[Starlark] Could not read config for a GitHub token", exc_info=True)
return None
def _get_tronbyte_repository_class() -> Type[Any]:
"""Import TronbyteRepository from plugin-repos directory."""
import importlib.util
import importlib
def _import_plugin_code_in_web_process(module_name: str, module_path: Path,
reuse: bool = True) -> Any:
"""Import a file of plugin code into the web process and return the module.
The only place the web process executes plugin code. Plugins run in the
display process; the web process reads them as files (PluginCatalog) and
runs a web-UI action's script as a subprocess. Two features still need a
plugin's Python in-process, and both come through here:
- Starlark: the standalone routes use the starlark-apps plugin's
``tronbyte_repository`` (browsing the app repository) and
``pixlet_renderer`` (rendering an app) -- helper modules, never the
plugin class itself.
- A web-UI action with ``oauth_flow``: step 1 calls the action script's
``get_auth_url()`` (or the Spotify credential helpers).
Temporary: the plugin web-entry contract (docs/ARCHITECTURE.md, "Web and
display processes") replaces both with an explicit, declared entry point
for plugin web code.
``reuse`` returns the module already imported under ``module_name``
instead of executing the file again. A module that fails to execute is
removed from sys.modules, so one transient failure cannot leave a
half-initialised module cached for the rest of the process (it used to
surface as AttributeError, not ImportError).
"""
import importlib.util
if reuse and module_name in sys.modules:
return sys.modules[module_name]
spec = importlib.util.spec_from_file_location(module_name, str(module_path))
if spec is None or spec.loader is None:
raise ImportError(f"Failed to create module spec for {module_name} at {module_path}")
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
try:
spec.loader.exec_module(module)
except BaseException:
sys.modules.pop(module_name, None)
raise
return module
def _get_tronbyte_repository_class() -> Type[Any]:
"""TronbyteRepository, from the installed starlark-apps plugin."""
module_path = PROJECT_ROOT / 'plugin-repos' / 'starlark-apps' / 'tronbyte_repository.py'
if not module_path.exists():
raise ImportError(f"TronbyteRepository module not found at {module_path}")
return _import_plugin_code_in_web_process('tronbyte_repository', module_path).TronbyteRepository
# If already imported, return cached class
if "tronbyte_repository" in sys.modules:
return sys.modules["tronbyte_repository"].TronbyteRepository
spec = importlib.util.spec_from_file_location("tronbyte_repository", str(module_path))
if spec is None:
raise ImportError(f"Failed to create module spec for tronbyte_repository at {module_path}")
module = importlib.util.module_from_spec(spec)
if module is None:
raise ImportError("Failed to create module from spec for tronbyte_repository")
sys.modules["tronbyte_repository"] = module
try:
spec.loader.exec_module(module)
except BaseException:
# A module that failed to execute must not stay in sys.modules: the
# cache branch above would hand back the half-initialised object for
# the rest of the process, so one transient failure would disable
# this path permanently and surface as AttributeError, not ImportError.
sys.modules.pop("tronbyte_repository", None)
raise
return module.TronbyteRepository
def _get_pixlet_renderer_class() -> Type[Any]:
"""Import PixletRenderer from plugin-repos directory."""
import importlib.util
import importlib
"""PixletRenderer, from the installed starlark-apps plugin."""
module_path = PROJECT_ROOT / 'plugin-repos' / 'starlark-apps' / 'pixlet_renderer.py'
if not module_path.exists():
raise ImportError(f"PixletRenderer module not found at {module_path}")
return _import_plugin_code_in_web_process('pixlet_renderer', module_path).PixletRenderer
# If already imported, return cached class
if "pixlet_renderer" in sys.modules:
return sys.modules["pixlet_renderer"].PixletRenderer
spec = importlib.util.spec_from_file_location("pixlet_renderer", str(module_path))
if spec is None:
raise ImportError(f"Failed to create module spec for pixlet_renderer at {module_path}")
module = importlib.util.module_from_spec(spec)
if module is None:
raise ImportError("Failed to create module from spec for pixlet_renderer")
sys.modules["pixlet_renderer"] = module
try:
spec.loader.exec_module(module)
except BaseException:
# A module that failed to execute must not stay in sys.modules: the
# cache branch above would hand back the half-initialised object for
# the rest of the process, so one transient failure would disable
# this path permanently and surface as AttributeError, not ImportError.
sys.modules.pop("pixlet_renderer", None)
raise
return module.PixletRenderer
def _validate_and_sanitize_app_id(app_id: Optional[str], fallback_source: Optional[str] = None) -> Tuple[Optional[str], Optional[str]]:
"""Validate and sanitize app_id to a safe slug."""
if not app_id and fallback_source:
web_interface/blueprints/api_v3/config.py
+9 -15
View File
@@ -1014,7 +1014,7 @@ def save_main_config():
plugin_manifests = _pkg._discovered_plugin_manifests()
for key in data:
# Check if this key is a plugin ID
if api_v3.plugin_manager and key in plugin_manifests:
if api_v3.plugin_catalog and key in plugin_manifests:
plugin_id = key
submitted_config = data[key]
if not isinstance(submitted_config, dict):
@@ -1030,7 +1030,7 @@ def save_main_config():
# the schema load are far enough apart that a later edit could
# separate them. Refuse rather than save without knowing which
# fields are secrets.
schema_path = resolve_under(api_v3.plugin_manager.plugins_dir,
schema_path = resolve_under(api_v3.plugin_catalog.plugins_dir,
plugin_id, 'config_schema.json')
if schema_path is None:
return error_response(
@@ -1116,18 +1116,9 @@ def save_main_config():
invalidate_cache()
# Notify saved plugins of their new config (with secrets merged), now
# that it is on disk.
for plugin_id in plugin_keys_to_remove:
try:
plugin_instance = api_v3.plugin_manager.get_plugin(plugin_id)
if plugin_instance and hasattr(plugin_instance, 'on_config_change'):
merged_config = api_v3.config_manager.load_config()
plugin_instance.on_config_change(_pkg._prepared_plugin_config(
plugin_id, merged_config.get(plugin_id, {})))
except Exception as hook_err:
# Don't fail the save if hook fails
logger.warning("on_config_change failed: %s", hook_err)
# Saved plugin sections reach the running plugins through the display
# process's config watcher (on_config_change there); nothing runs a
# plugin in this process.
message = 'Configuration saved successfully'
# Switching automatic updates on finishes their setup, which needs
@@ -1139,7 +1130,10 @@ def save_main_config():
message = f'{message}. {note}'
except Exception:
logger.warning("Automatic update setup could not be started", exc_info=True)
return success_response(message=message)
# Display hardware, rotation/durations and general settings take
# effect after a display restart; the UI shows its restart banner on
# this flag.
return success_response(message=message, extra={'restart_required': True})
except Exception as e:
logger.error("Error saving config", exc_info=True)
return error_response(
web_interface/blueprints/api_v3/display.py
+9 -9
View File
@@ -77,19 +77,19 @@ def get_display_modes():
for the duration -- so they are reported with enabled: false
rather than omitted.
"""
if not api_v3.plugin_manager:
return jsonify({'status': 'error', 'message': 'Plugin manager not initialized'}), 500
if not api_v3.plugin_catalog:
return jsonify({'status': 'error', 'message': 'Plugin catalog not initialized'}), 500
# Discovery is lazy and normally triggered by whichever endpoint runs
# first, which is a person opening the dashboard. A caller that never
# visits it would otherwise see an empty list.
api_v3.plugin_manager.discover_plugins()
api_v3.plugin_catalog.discover_plugins()
include_disabled = request.args.get('include_disabled') in ('1', 'true', 'True')
full_config = api_v3.config_manager.load_config() if api_v3.config_manager else {}
modes = []
for plugin_id, manifest in sorted(api_v3.plugin_manager.plugin_manifests.items()):
for plugin_id, manifest in sorted(api_v3.plugin_catalog.plugin_manifests.items()):
# A hand-edited or migrated config.json can hold a non-dict under a
# plugin id; DisplayController._reconcile guards the same shape, so
# it happens in practice. Without this, .get() raises AttributeError,
@@ -107,7 +107,7 @@ def get_display_modes():
if not enabled and not include_disabled:
continue
plugin_name = (manifest or {}).get('name') or plugin_id
plugin_modes = api_v3.plugin_manager.get_plugin_display_modes(plugin_id) or [plugin_id]
plugin_modes = api_v3.plugin_catalog.get_plugin_display_modes(plugin_id) or [plugin_id]
for mode in plugin_modes:
# A single-mode plugin's mode is the plugin, so its own name is
# the readable label. Multi-mode plugins have no per-mode name
@@ -162,21 +162,21 @@ def start_on_demand_display():
resolved_plugin = plugin_id
resolved_mode = mode
if api_v3.plugin_manager:
if api_v3.plugin_catalog:
if resolved_plugin and resolved_plugin not in _pkg._discovered_plugin_manifests(resolved_plugin):
return jsonify({'status': 'error', 'message': f'Plugin {resolved_plugin} not found'}), 404
if resolved_plugin and not resolved_mode:
modes = api_v3.plugin_manager.get_plugin_display_modes(resolved_plugin)
modes = api_v3.plugin_catalog.get_plugin_display_modes(resolved_plugin)
resolved_mode = modes[0] if modes else resolved_plugin
elif resolved_mode and not resolved_plugin:
_pkg._discovered_plugin_manifests()
resolved_plugin = api_v3.plugin_manager.find_plugin_for_mode(resolved_mode)
resolved_plugin = api_v3.plugin_catalog.find_plugin_for_mode(resolved_mode)
if not resolved_plugin:
# Not among what was discovered: the plugin that declares
# it may have been installed since. Scan once more.
_pkg._discovered_plugin_manifests(rescan=True)
resolved_plugin = api_v3.plugin_manager.find_plugin_for_mode(resolved_mode)
resolved_plugin = api_v3.plugin_catalog.find_plugin_for_mode(resolved_mode)
if not resolved_plugin:
return jsonify({'status': 'error', 'message': f'Mode {resolved_mode} not found'}), 404
web_interface/blueprints/api_v3/misc.py
+1 -1
View File
@@ -76,7 +76,7 @@ def get_health():
# Check plugin system
try:
if api_v3.plugin_manager:
if api_v3.plugin_catalog:
plugin_count = len(_discovered_plugin_manifests())
health_status['checks']['plugin_system'] = {
'status': 'operational',
web_interface/blueprints/api_v3/plugin_config.py
+6 -52
View File
@@ -452,40 +452,9 @@ def save_plugin_config():
status_code=500
)
# If the plugin is loaded, notify it of the config change with merged config
try:
if api_v3.plugin_manager:
plugin_instance = api_v3.plugin_manager.get_plugin(plugin_id)
if plugin_instance:
# Reload merged config (includes secrets) and pass the plugin-specific section
merged_config = api_v3.config_manager.load_config()
plugin_full_config = _pkg._prepared_plugin_config(
plugin_id, merged_config.get(plugin_id, {}))
if hasattr(plugin_instance, 'on_config_change'):
plugin_instance.on_config_change(plugin_full_config)
# Update plugin state manager and call lifecycle methods based on enabled state
# This ensures the plugin state is synchronized with the config
enabled = plugin_full_config.get('enabled', plugin_instance.enabled)
# Update state manager if available
if api_v3.plugin_state_manager:
api_v3.plugin_state_manager.set_plugin_enabled(plugin_id, enabled)
# Call lifecycle methods to ensure plugin state matches config
try:
if enabled:
if hasattr(plugin_instance, 'on_enable'):
plugin_instance.on_enable()
else:
if hasattr(plugin_instance, 'on_disable'):
plugin_instance.on_disable()
except Exception as lifecycle_error:
# Log the error but don't fail the save - config is already saved
logger.warning("Lifecycle method error for %s: %s", plugin_id, lifecycle_error, exc_info=True)
except Exception as hook_err:
# Do not fail the save if hook fails; just log
logger.warning("on_config_change failed: %s", hook_err)
# The running plugin hears about this from the display process: its
# config watcher calls on_config_change with the prepared section, and
# loads or unloads the plugin if "enabled" changed.
secret_count = len(secrets_config)
message = f'Plugin {plugin_id} configuration saved successfully'
@@ -543,12 +512,7 @@ def _prepare_plugin_config_for_save(plugin_id, plugin_config, schema, schema_mgr
current_config = api_v3.config_manager.load_config()
if plugin_id in current_config and 'enabled' in current_config[plugin_id]:
plugin_config['enabled'] = current_config[plugin_id]['enabled']
elif api_v3.plugin_manager:
# Fallback to plugin instance if config doesn't have it
plugin_instance = api_v3.plugin_manager.get_plugin(plugin_id)
if plugin_instance:
plugin_config['enabled'] = plugin_instance.enabled
# Final fallback: default to True if plugin is loaded (matches BasePlugin default)
# Fallback: default to True (matches BasePlugin default)
if 'enabled' not in plugin_config:
plugin_config['enabled'] = True
except Exception as e:
@@ -977,18 +941,8 @@ def reset_plugin_config():
if default_secrets or not preserve_secrets:
api_v3.config_manager.save_raw_file_content('secrets', current_secrets)
# Notify plugin of config change if loaded
try:
if api_v3.plugin_manager:
plugin_instance = api_v3.plugin_manager.get_plugin(plugin_id)
if plugin_instance:
merged_config = api_v3.config_manager.load_config()
plugin_full_config = _pkg._prepared_plugin_config(
plugin_id, merged_config.get(plugin_id, {}))
if hasattr(plugin_instance, 'on_config_change'):
plugin_instance.on_config_change(plugin_full_config)
except Exception as hook_err:
logger.warning("on_config_change failed: %s", hook_err)
# The display's config watcher passes the reset config to the running
# plugin (on_config_change); nothing to notify in this process.
return jsonify({
'status': 'success',
web_interface/blueprints/api_v3/plugin_health.py
+43 -31
View File
@@ -1,7 +1,9 @@
"""Plugin health, resource metrics and resource limits.
These read and reset the web process's own trackers; the display service
keeps its own (see the route docstrings).
The display process records health and metrics to the shared on-disk cache;
these routes read (and reset) that published state through a tracker and a
monitor backed by the same cache (app.py sets api_v3.health_tracker and
api_v3.resource_monitor). See the route docstrings.
Routes decorate the shared `api_v3` Blueprint from the package `__init__`,
so their endpoint names do not depend on which module they live in.
@@ -11,20 +13,30 @@ from web_interface.blueprints.api_v3 import (
)
def _health_tracker():
"""The reader of the display's published plugin health, or None."""
return getattr(api_v3, 'health_tracker', None)
def _resource_monitor():
"""The reader of the display's published plugin metrics, or None."""
return getattr(api_v3, 'resource_monitor', None)
@api_v3.route('/plugins/health', methods=['GET'])
def get_plugin_health():
"""Get health metrics for all plugins"""
if not api_v3.plugin_manager:
return jsonify({'status': 'error', 'message': 'Plugin manager not initialized'}), 500
if not api_v3.plugin_catalog:
return jsonify({'status': 'error', 'message': 'Plugin catalog not initialized'}), 500
if not api_v3.plugin_manager.health_tracker:
if not _health_tracker():
return jsonify({
'status': 'success',
'data': {},
'message': 'Health tracking not available'
})
tracker = api_v3.plugin_manager.health_tracker
tracker = _health_tracker()
# Build per-plugin summaries by ID so persisted (cross-process) health
# is included, then fold in any in-memory-only entries.
health_summaries = {}
@@ -51,17 +63,17 @@ def get_plugin_health():
@api_v3.route('/plugins/health/<plugin_id>', methods=['GET'])
def get_plugin_health_single(plugin_id):
"""Get health metrics for a specific plugin"""
if not api_v3.plugin_manager:
return jsonify({'status': 'error', 'message': 'Plugin manager not initialized'}), 500
if not api_v3.plugin_catalog:
return jsonify({'status': 'error', 'message': 'Plugin catalog not initialized'}), 500
if not api_v3.plugin_manager.health_tracker:
if not _health_tracker():
return jsonify({
'status': 'error',
'message': 'Health tracking not available'
}), 503
# force_reload for the same reason as the list route above.
health_summary = api_v3.plugin_manager.health_tracker.get_health_summary(
health_summary = _health_tracker().get_health_summary(
plugin_id, force_reload=True)
return jsonify({
@@ -79,17 +91,17 @@ def reset_plugin_health(plugin_id):
in-memory state, so its next recorded success or failure can write that
state back; restart the display service for a reset it will honour.
"""
if not api_v3.plugin_manager:
return jsonify({'status': 'error', 'message': 'Plugin manager not initialized'}), 500
if not api_v3.plugin_catalog:
return jsonify({'status': 'error', 'message': 'Plugin catalog not initialized'}), 500
if not api_v3.plugin_manager.health_tracker:
if not _health_tracker():
return jsonify({
'status': 'error',
'message': 'Health tracking not available'
}), 503
# Reset health state
api_v3.plugin_manager.health_tracker.reset_health(plugin_id)
_health_tracker().reset_health(plugin_id)
return jsonify({
'status': 'success',
@@ -100,17 +112,17 @@ def reset_plugin_health(plugin_id):
@api_v3.route('/plugins/metrics', methods=['GET'])
def get_plugin_metrics():
"""Get resource metrics for all plugins"""
if not api_v3.plugin_manager:
return jsonify({'status': 'error', 'message': 'Plugin manager not initialized'}), 500
if not api_v3.plugin_catalog:
return jsonify({'status': 'error', 'message': 'Plugin catalog not initialized'}), 500
if not api_v3.plugin_manager.resource_monitor:
if not _resource_monitor():
return jsonify({
'status': 'success',
'data': {},
'message': 'Resource monitoring not available'
})
monitor = api_v3.plugin_manager.resource_monitor
monitor = _resource_monitor()
# Build per-plugin summaries by ID so persisted (cross-process) metrics
# are included, then fold in any in-memory-only entries.
metrics_summaries = {}
@@ -136,17 +148,17 @@ def get_plugin_metrics():
@api_v3.route('/plugins/metrics/<plugin_id>', methods=['GET'])
def get_plugin_metrics_single(plugin_id):
"""Get resource metrics for a specific plugin"""
if not api_v3.plugin_manager:
return jsonify({'status': 'error', 'message': 'Plugin manager not initialized'}), 500
if not api_v3.plugin_catalog:
return jsonify({'status': 'error', 'message': 'Plugin catalog not initialized'}), 500
if not api_v3.plugin_manager.resource_monitor:
if not _resource_monitor():
return jsonify({
'status': 'error',
'message': 'Resource monitoring not available'
}), 503
# force_reload for the same reason as the list route above.
metrics_summary = api_v3.plugin_manager.resource_monitor.get_metrics_summary(
metrics_summary = _resource_monitor().get_metrics_summary(
plugin_id, force_reload=True)
return jsonify({
@@ -163,17 +175,17 @@ def reset_plugin_metrics(plugin_id):
display service keeps accumulating in its own process and republishes
its totals on its next persist, so the reset does not stick while it runs.
"""
if not api_v3.plugin_manager:
return jsonify({'status': 'error', 'message': 'Plugin manager not initialized'}), 500
if not api_v3.plugin_catalog:
return jsonify({'status': 'error', 'message': 'Plugin catalog not initialized'}), 500
if not api_v3.plugin_manager.resource_monitor:
if not _resource_monitor():
return jsonify({
'status': 'error',
'message': 'Resource monitoring not available'
}), 503
# Reset metrics
api_v3.plugin_manager.resource_monitor.reset_metrics(plugin_id)
_resource_monitor().reset_metrics(plugin_id)
return jsonify({
'status': 'success',
@@ -190,10 +202,10 @@ def manage_plugin_limits(plugin_id):
plugin, so a change to existing limits takes effect there after the
display service restarts.
"""
if not api_v3.plugin_manager:
return jsonify({'status': 'error', 'message': 'Plugin manager not initialized'}), 500
if not api_v3.plugin_catalog:
return jsonify({'status': 'error', 'message': 'Plugin catalog not initialized'}), 500
if not api_v3.plugin_manager.resource_monitor:
if not _resource_monitor():
return jsonify({
'status': 'error',
'message': 'Resource monitoring not available'
@@ -201,7 +213,7 @@ def manage_plugin_limits(plugin_id):
if request.method == 'GET':
# Get limits
limits = api_v3.plugin_manager.resource_monitor.get_limits(plugin_id)
limits = _resource_monitor().get_limits(plugin_id)
if limits:
return jsonify({
'status': 'success',
@@ -234,7 +246,7 @@ def manage_plugin_limits(plugin_id):
'message': f'{bad} must be a non-negative number or null'}), 400
limits = limits_from_dict(data)
api_v3.plugin_manager.resource_monitor.set_limits(plugin_id, limits)
_resource_monitor().set_limits(plugin_id, limits)
return jsonify({
'status': 'success',
web_interface/blueprints/api_v3/plugin_operations.py
+5 -5
View File
@@ -120,10 +120,10 @@ def get_plugin_state():
def reconcile_plugin_state():
"""Reconcile plugin state across all sources"""
try:
if not api_v3.plugin_state_manager or not api_v3.plugin_manager:
if not api_v3.plugin_state_manager or not api_v3.plugin_catalog:
return error_response(
ErrorCode.SYSTEM_ERROR,
'State manager or plugin manager not initialized',
'State manager or plugin catalog not initialized',
status_code=500
)
@@ -139,8 +139,8 @@ def reconcile_plugin_state():
reconciler = StateReconciliation(
state_manager=api_v3.plugin_state_manager,
config_manager=api_v3.config_manager,
plugin_manager=api_v3.plugin_manager,
plugins_dir=Path(api_v3.plugin_manager.plugins_dir)
plugin_manager=api_v3.plugin_catalog,
plugins_dir=Path(api_v3.plugin_catalog.plugins_dir)
)
result = reconciler.reconcile_state(force=force)
@@ -205,7 +205,7 @@ def _drop_stale_reconciliation_findings(unresolved):
)
cm = api_v3.config_manager
plugins_dir = getattr(api_v3.plugin_manager, 'plugins_dir', None)
plugins_dir = getattr(api_v3.plugin_catalog, 'plugins_dir', None)
installed = disk_plugin_ids(plugins_dir) if plugins_dir else set()
config_keys = config_plugin_ids(cm.load_config() or {},
ignored_config_keys(cm, installed))
web_interface/blueprints/api_v3/plugin_store.py
+41 -22
View File
@@ -6,7 +6,8 @@ so their endpoint names do not depend on which module they live in.
"""
from web_interface.blueprints.api_v3 import (
ErrorCode, OperationType, Path, _do_transactional_uninstall,
_non_plugin_id_error, _get_plugin_version, _plugin_directory, api_v3,
_non_plugin_id_error, _get_plugin_version, _plugin_directory,
_plugin_enabled_in_config, _store_restart_fields, api_v3,
datetime, error_response, exception_error_response, json, jsonify, logger,
request, success_response, validate_request_json,
)
@@ -228,11 +229,12 @@ def update_plugin():
if api_v3.schema_manager:
api_v3.schema_manager.invalidate_cache(plugin_id)
# Rediscover plugins
if api_v3.plugin_manager:
api_v3.plugin_manager.discover_plugins()
if plugin_id in api_v3.plugin_manager.plugins:
api_v3.plugin_manager.reload_plugin(plugin_id)
# Rediscover plugins. The web process runs no plugin code, so
# there is nothing here to reload: the display keeps running the
# version it loaded until it restarts, which restart_required
# below asks for.
if api_v3.plugin_catalog:
api_v3.plugin_catalog.discover_plugins()
# Update state and history
if api_v3.plugin_state_manager:
@@ -261,7 +263,10 @@ def update_plugin():
'commit': updated_commit,
'update_status': update_status
},
message=message
message=message,
extra=_store_restart_fields(
'update', _plugin_enabled_in_config(plugin_id),
changed=update_status == 'updated'),
)
else:
refusal = _compatibility_refusal(plugin_id)
@@ -341,6 +346,8 @@ def uninstall_plugin():
if api_v3.operation_queue:
def uninstall_callback(operation):
"""Callback to execute plugin uninstallation via transactional helper."""
# Read before the uninstall removes the config section.
was_enabled = _plugin_enabled_in_config(plugin_id)
success, error_msg = _do_transactional_uninstall(plugin_id, preserve_config)
if not success:
if api_v3.operation_history:
@@ -358,7 +365,9 @@ def uninstall_plugin():
status="success",
details={"preserve_config": preserve_config}
)
return {'success': True, 'message': 'Plugin uninstalled successfully'}
return {'success': True, 'message': 'Plugin uninstalled successfully',
**_store_restart_fields('uninstall', was_enabled,
preserve_config=preserve_config)}
# Enqueue operation
operation_id = api_v3.operation_queue.enqueue_operation(
@@ -373,6 +382,7 @@ def uninstall_plugin():
)
else:
# Direct (non-queued) transactional uninstall
was_enabled = _plugin_enabled_in_config(plugin_id)
success, error_msg = _do_transactional_uninstall(plugin_id, preserve_config)
if success:
@@ -383,7 +393,10 @@ def uninstall_plugin():
status="success",
details={"preserve_config": preserve_config}
)
return success_response(message='Plugin uninstalled successfully')
return success_response(
message='Plugin uninstalled successfully',
extra=_store_restart_fields('uninstall', was_enabled,
preserve_config=preserve_config))
else:
if api_v3.operation_history:
api_v3.operation_history.record_operation(
@@ -448,10 +461,11 @@ def install_plugin():
if api_v3.schema_manager:
api_v3.schema_manager.invalidate_cache(plugin_id)
# Discover and load the new plugin
if api_v3.plugin_manager:
api_v3.plugin_manager.discover_plugins()
api_v3.plugin_manager.load_plugin(plugin_id)
# List the new plugin. The display loads it, from disk, when
# it is enabled; see restart_required below for one that
# already is.
if api_v3.plugin_catalog:
api_v3.plugin_catalog.discover_plugins()
# Update state manager
if api_v3.plugin_state_manager:
@@ -468,7 +482,9 @@ def install_plugin():
)
branch_msg = f" (branch: {branch})" if branch else ""
return {'success': True, 'message': f'Plugin {plugin_id} installed successfully{branch_msg}'}
return {'success': True,
'message': f'Plugin {plugin_id} installed successfully{branch_msg}',
**_store_restart_fields('install', _plugin_enabled_in_config(plugin_id))}
else:
error_msg = f'Failed to install plugin {plugin_id}'
if branch:
@@ -511,9 +527,8 @@ def install_plugin():
if success:
if api_v3.schema_manager:
api_v3.schema_manager.invalidate_cache(plugin_id)
if api_v3.plugin_manager:
api_v3.plugin_manager.discover_plugins()
api_v3.plugin_manager.load_plugin(plugin_id)
if api_v3.plugin_catalog:
api_v3.plugin_catalog.discover_plugins()
if api_v3.plugin_state_manager:
api_v3.plugin_state_manager.set_plugin_installed(plugin_id)
if api_v3.operation_history:
@@ -526,7 +541,9 @@ def install_plugin():
)
branch_msg = f" (branch: {branch})" if branch else ""
return success_response(message=f'Plugin installed successfully{branch_msg}')
return success_response(
message=f'Plugin installed successfully{branch_msg}',
extra=_store_restart_fields('install', _plugin_enabled_in_config(plugin_id)))
else:
error_msg = f'Failed to install plugin {plugin_id}'
if branch:
@@ -587,10 +604,9 @@ def install_plugin_from_url():
if api_v3.schema_manager and installed_plugin_id:
api_v3.schema_manager.invalidate_cache(installed_plugin_id)
# Discover and load the new plugin
if api_v3.plugin_manager and installed_plugin_id:
api_v3.plugin_manager.discover_plugins()
api_v3.plugin_manager.load_plugin(installed_plugin_id)
# List the new plugin; the display loads it when it is enabled.
if api_v3.plugin_catalog and installed_plugin_id:
api_v3.plugin_catalog.discover_plugins()
branch_msg = f" (branch: {result.get('branch', branch)})" if (result.get('branch') or branch) else ""
response_data = {
@@ -601,6 +617,9 @@ def install_plugin_from_url():
}
if result.get('branch'):
response_data['branch'] = result.get('branch')
if installed_plugin_id:
response_data.update(_store_restart_fields(
'install', _plugin_enabled_in_config(installed_plugin_id)))
return jsonify(response_data)
else:
return jsonify({
web_interface/blueprints/api_v3/plugins.py
+64 -80
View File
@@ -11,7 +11,7 @@ from web_interface.blueprints.api_v3 import (
)
from src.common.path_safety import safe_path_component
from src.plugin_system.base_plugin import (
configured_vegas_participation, resolve_vegas_participation,
configured_vegas_participation, vegas_participation_value,
)
import web_interface.blueprints.api_v3 as _pkg
# Read through the module rather than bound by value: tests patch these
@@ -20,18 +20,48 @@ import web_interface.blueprints.api_v3 as _pkg
# package is the only patch point that covers every caller.
def _vegas_participation(plugin_id, plugin_config, manifest):
"""What Vegas does with a plugin, as far as its files say, and from where.
The order the display resolves it in (resolve_vegas_participation), up
to where that needs the plugin's code: the user's ``vegas_participation`` setting
(``'config'``), then the manifest's declared ``vegas_participation``
(``'manifest'``). Past those the display asks the plugin itself -- a
get_vegas_participation() override or the legacy Vegas hooks -- which the
web process never runs, so the answer is ``(None, 'runtime')``: decided
at run time, not guessed here. A plugin that overrides
get_vegas_participation() can still differ from its manifest.
"""
configured = configured_vegas_participation(plugin_id, plugin_config)
if configured is not None:
return configured, 'config'
declared = vegas_participation_value(
manifest.get('vegas_participation') if isinstance(manifest, dict) else None)
if declared is not None:
return declared, 'manifest'
return None, 'runtime'
@api_v3.route('/plugins/installed', methods=['GET'])
def get_installed_plugins():
"""Get installed plugins"""
if not api_v3.plugin_manager or not api_v3.plugin_store_manager:
"""Get installed plugins.
Metadata comes from the plugin catalog (manifests on disk), ``enabled``
from config.json. ``loaded``, ``state`` and ``error_info`` are always
null: they would describe the display process's plugin instances, and
the display does not publish which plugins it has loaded. What it does
publish -- health, metrics, errors -- is served by /plugins/health,
/plugins/metrics and /errors.
"""
if not api_v3.plugin_catalog or not api_v3.plugin_store_manager:
return jsonify({'status': 'error', 'message': 'Plugin managers not initialized'}), 500
# Re-discover plugins to ensure we have the latest list
# This handles cases where plugins are added/removed after app startup
api_v3.plugin_manager.discover_plugins()
api_v3.plugin_catalog.discover_plugins()
# Get all installed plugin info from the plugin manager
all_plugin_info = api_v3.plugin_manager.get_all_plugin_info()
# Get all installed plugin info from the catalog
all_plugin_info = api_v3.plugin_catalog.get_all_plugin_info()
# Load config once before the loop (not per-plugin)
full_config = api_v3.config_manager.load_config() if api_v3.config_manager else {}
@@ -45,18 +75,6 @@ def get_installed_plugins():
return None
def _build_plugin_entry_inner(plugin_info, plugin_id):
# Capture runtime state (state machine + error context) before the
# manifest merge below can shadow the 'state' key. get_all_plugin_info
# attaches this via PluginStateManager.get_state_info(); surfacing it
# lets the UI show *why* a plugin isn't running instead of just
# 'loaded: false'.
state_info = plugin_info.get('state')
plugin_state = None
plugin_error_info = None
if isinstance(state_info, dict):
plugin_state = state_info.get('state')
plugin_error_info = state_info.get('error_info')
# Re-read manifest from disk to ensure we have the latest metadata.
# Through the resolver, not plugins_dir/<id>: a plugin installed as
# ledmatrix-<id> otherwise never had its manifest refreshed here.
@@ -74,16 +92,13 @@ def get_installed_plugins():
except (FileNotFoundError, PermissionError, json.JSONDecodeError) as e:
logger.debug("Could not read fresh manifest for %s: %s", plugin_id, e)
# Enabled status: config is source of truth, fall back to instance
enabled = None
# Enabled status: config.json, read by the display's rule -- it runs
# a plugin only when its section says "enabled": true, so a missing
# flag is disabled here too.
plugin_config = full_config.get(plugin_id, {})
if 'enabled' in plugin_config:
enabled = bool(plugin_config['enabled'])
# Single get_plugin() call shared for both enabled fallback and Vegas mode
plugin_instance = api_v3.plugin_manager.get_plugin(plugin_id)
if enabled is None:
enabled = plugin_instance.enabled if plugin_instance else True
if not isinstance(plugin_config, dict):
plugin_config = {}
enabled = bool(plugin_config.get('enabled', False))
# Verified + latest published version from registry (no network call)
store_info = api_v3.plugin_store_manager.get_registry_info(plugin_id)
@@ -113,32 +128,16 @@ def get_installed_plugins():
if store_info and not last_commit_message:
last_commit_message = store_info.get('last_commit_message')
# Vegas mode from instance, overridden by explicit config value
vegas_mode = None
# Vegas mode as configured. What a plugin's code would choose on its
# own is only known to the display, which runs it.
vegas_mode = plugin_config.get('vegas_mode')
vegas_content_type = None
if plugin_instance:
try:
if hasattr(plugin_instance, 'get_vegas_display_mode'):
mode = plugin_instance.get_vegas_display_mode()
vegas_mode = mode.value if hasattr(mode, 'value') else str(mode)
except (AttributeError, TypeError, ValueError) as e:
logger.debug("[%s] Failed to get vegas_display_mode: %s", plugin_id, e)
try:
if hasattr(plugin_instance, 'get_vegas_content_type'):
vegas_content_type = plugin_instance.get_vegas_content_type()
except (AttributeError, TypeError, ValueError) as e:
logger.debug("[%s] Failed to get vegas_content_type: %s", plugin_id, e)
if 'vegas_mode' in plugin_config:
vegas_mode = plugin_config['vegas_mode']
# What Vegas actually does with the plugin: 'scroll', 'pause' or
# 'exclude'. The same resolution the ticker uses; without a loaded
# instance only the user's own setting is known.
if plugin_instance is not None:
vegas_participation = resolve_vegas_participation(plugin_instance, plugin_id)
else:
vegas_participation = configured_vegas_participation(plugin_id, plugin_config)
# What Vegas does with it: 'scroll', 'pause' or 'exclude', or None
# when only the plugin's code (run by the display) decides. The Vegas
# order list badges a None as its configured vegas_mode, else Scroll.
vegas_participation, vegas_participation_source = _vegas_participation(
plugin_id, plugin_config, plugin_info)
return {
'id': plugin_id,
@@ -155,9 +154,10 @@ def get_installed_plugins():
'icon': plugin_info.get('icon') if isinstance(plugin_info.get('icon'), str) else None,
'enabled': enabled,
'verified': verified,
'loaded': plugin_info.get('loaded', False),
'state': plugin_state,
'error_info': plugin_error_info,
# Not published by the display process; see the docstring.
'loaded': None,
'state': None,
'error_info': None,
'last_updated': last_updated,
'last_commit': last_commit,
'last_commit_message': last_commit_message,
@@ -166,6 +166,7 @@ def get_installed_plugins():
'vegas_mode': vegas_mode,
'vegas_content_type': vegas_content_type,
'vegas_participation': vegas_participation,
'vegas_participation_source': vegas_participation_source,
}
from concurrent.futures import ThreadPoolExecutor
@@ -183,7 +184,7 @@ def toggle_plugin():
plugin_id = None
enabled = None
try:
if not api_v3.plugin_manager or not api_v3.config_manager:
if not api_v3.plugin_catalog or not api_v3.config_manager:
return jsonify({'status': 'error', 'message': 'Plugin or config manager not initialized'}), 500
# Support both JSON and form data (for HTMX submissions)
@@ -221,7 +222,7 @@ def toggle_plugin():
current_enabled = config.get(plugin_id, {}).get('enabled', False)
enabled = not current_enabled
# A Starlark app is not a plugin in plugin_manager's sense -- it is an
# A Starlark app is not a plugin in the catalog's sense -- it is an
# entry in starlark-apps' own manifest -- so its enable/disable is
# handled here rather than falling through to the check below, which
# would answer "Plugin not found".
@@ -258,21 +259,9 @@ def toggle_plugin():
status="success"
)
# If plugin is loaded, also call its lifecycle methods
# Wrap in try/except to prevent lifecycle errors from failing the toggle
plugin = api_v3.plugin_manager.get_plugin(plugin_id)
if plugin:
try:
if enabled:
if hasattr(plugin, 'on_enable'):
plugin.on_enable()
else:
if hasattr(plugin, 'on_disable'):
plugin.on_disable()
except Exception as lifecycle_error:
# Log the error but don't fail the toggle - config is already saved
logger.warning("Lifecycle method error for %s: %s", plugin_id, lifecycle_error, exc_info=True)
# No lifecycle hooks here: the display's config watcher sees the
# enabled flag change and loads or unloads the plugin itself
# (DisplayController._reconcile_enabled_plugins).
return success_response(
message=f"Plugin {plugin_id} {'enabled' if enabled else 'disabled'} successfully"
)
@@ -511,16 +500,11 @@ sys.exit(proc.returncode)
# Step 1: Get initial data (like auth URL)
# For OAuth flows, we might need to import the script as a module
if action_def.get('oauth_flow'):
# Import script as module to get auth URL
import sys
import importlib.util
spec = importlib.util.spec_from_file_location("plugin_action", script_file)
action_module = importlib.util.module_from_spec(spec)
sys.modules["plugin_action"] = action_module
try:
spec.loader.exec_module(action_module)
# Plugin code in the web process: see the
# function for why, and what replaces it.
action_module = _pkg._import_plugin_code_in_web_process(
"plugin_action", script_file, reuse=False)
# Try to get auth URL using common patterns
auth_url = None
web_interface/blueprints/api_v3/system.py
+1 -1
View File
@@ -506,7 +506,7 @@ def execute_system_action():
'output': "\n".join(outputs)
})
elif action == 'install_plugin_requirements':
active_pm = getattr(api_v3, 'plugin_manager', None)
active_pm = getattr(api_v3, 'plugin_catalog', None)
if active_pm:
plugins_dir = Path(active_pm.plugins_dir)
else:
web_interface/blueprints/pages_v3.py
+25 -26
View File
@@ -20,7 +20,8 @@ from web_interface import widget_bundle
logger = logging.getLogger(__name__)
# The managers live on the blueprint object: app.py sets
# pages_v3.config_manager, pages_v3.plugin_manager and the rest.
# pages_v3.config_manager, pages_v3.plugin_catalog and the rest. The catalog
# reads plugins as files; this process never runs plugin code.
pages_v3 = Blueprint('pages_v3', __name__)
@@ -201,11 +202,11 @@ def settings_search_index():
]
try:
plugin_ids = []
if pages_v3.plugin_manager:
if pages_v3.plugin_catalog:
try:
pages_v3.plugin_manager.discover_plugins()
pages_v3.plugin_catalog.discover_plugins()
plugin_ids = sorted(
pi.get('id') for pi in pages_v3.plugin_manager.get_all_plugin_info()
pi.get('id') for pi in pages_v3.plugin_catalog.get_all_plugin_info()
if pi.get('id')
)
except Exception:
@@ -220,7 +221,7 @@ def settings_search_index():
fields.extend(_extract_settings_fields(_partial_html(loader), tab, label))
for pid in plugin_ids:
info = pages_v3.plugin_manager.get_plugin_info(pid) or {}
info = pages_v3.plugin_catalog.get_plugin_info(pid) or {}
label = info.get('name', pid)
html = _partial_html(lambda pid=pid: _load_plugin_config_partial(pid))
fields.extend(_extract_settings_fields(html, pid, label))
@@ -264,8 +265,8 @@ def serve_plugin_web_ui(plugin_id, filename):
if not safe_id or not safe_fn:
return 'Invalid path component', 400, {'Content-Type': 'text/plain'}
if not pages_v3.plugin_manager:
return 'Plugin manager not available', 503, {'Content-Type': 'text/plain'}
if not pages_v3.plugin_catalog:
return 'Plugin catalog not available', 503, {'Content-Type': 'text/plain'}
try:
web_ui_path = resolve_under(_plugin_dir_for(safe_id) / 'web_ui', safe_fn)
@@ -329,7 +330,7 @@ def _resolved_plugin_dir(plugin_id):
its id is found there), and it refuses anything that is not one plain
path segment. See src/plugin_system/plugin_dirs.py.
"""
found = pages_v3.plugin_manager.get_plugin_directory(plugin_id)
found = pages_v3.plugin_catalog.get_plugin_directory(plugin_id)
if isinstance(found, (str, Path)) and Path(found).exists():
return Path(found)
return None
@@ -347,7 +348,7 @@ def _plugin_dir_for(safe_id):
if resolved is not None:
return resolved
plugins_base = Path(pages_v3.plugin_manager.plugins_dir).resolve()
plugins_base = Path(pages_v3.plugin_catalog.plugins_dir).resolve()
plugin_dir = resolve_under(plugins_base, safe_id)
if plugin_dir is None:
raise ValueError('plugin id escapes the plugins directory')
@@ -416,8 +417,8 @@ def serve_plugin_widget(plugin_id, widget_name):
if not safe_id or not safe_widget:
return 'Invalid path component', 400, {'Content-Type': 'text/plain'}
if not pages_v3.plugin_manager:
return 'Plugin manager not available', 503, {'Content-Type': 'text/plain'}
if not pages_v3.plugin_catalog:
return 'Plugin catalog not available', 503, {'Content-Type': 'text/plain'}
try:
plugin_dir = _plugin_dir_for(safe_id)
@@ -543,17 +544,17 @@ def _load_durations_partial():
main_config = pages_v3.config_manager.load_config()
duration_groups = []
covered_keys = set()
if pages_v3.plugin_manager:
if pages_v3.plugin_catalog:
try:
pages_v3.plugin_manager.discover_plugins()
pages_v3.plugin_catalog.discover_plugins()
saved = (main_config.get('display', {}) or {}).get('display_durations', {}) or {}
infos = sorted(pages_v3.plugin_manager.get_all_plugin_info(),
infos = sorted(pages_v3.plugin_catalog.get_all_plugin_info(),
key=lambda i: (i.get('name') or i.get('id') or '').lower())
for info in infos:
pid = info.get('id')
if not pid or not (main_config.get(pid, {}) or {}).get('enabled', False):
continue
modes = pages_v3.plugin_manager.get_plugin_display_modes(pid) or [pid]
modes = pages_v3.plugin_catalog.get_plugin_display_modes(pid) or [pid]
covered_keys.update(modes)
default = _plugin_default_duration(pid, main_config.get(pid, {}) or {})
duration_groups.append({
@@ -691,7 +692,7 @@ def _load_plugin_config_partial(plugin_id):
return '<div class="text-red-500 p-4">Invalid plugin ID</div>', 400
try:
if not pages_v3.plugin_manager:
if not pages_v3.plugin_catalog:
return '<div class="text-red-500 p-4">Plugin manager not available</div>', 500
# Handle starlark app config (starlark:<app_id>)
@@ -699,18 +700,18 @@ def _load_plugin_config_partial(plugin_id):
return _load_starlark_config_partial(plugin_id[len('starlark:'):])
# Resolve and validate all plugin paths against the plugins base directory
_plugins_base = Path(pages_v3.plugin_manager.plugins_dir).resolve()
_plugins_base = Path(pages_v3.plugin_catalog.plugins_dir).resolve()
_plugin_dir = resolve_under(_plugins_base, plugin_id)
if _plugin_dir is None:
return '<div class="text-red-500 p-4">Invalid plugin ID</div>', 400
# Try to get plugin info first
plugin_info = pages_v3.plugin_manager.get_plugin_info(plugin_id)
plugin_info = pages_v3.plugin_catalog.get_plugin_info(plugin_id)
# If not found, re-discover plugins (handles plugins added after startup)
if not plugin_info:
pages_v3.plugin_manager.discover_plugins()
plugin_info = pages_v3.plugin_manager.get_plugin_info(plugin_id)
pages_v3.plugin_catalog.discover_plugins()
plugin_info = pages_v3.plugin_catalog.get_plugin_info(plugin_id)
if not plugin_info:
return '<div class="text-red-500 p-4">Plugin not found</div>', 404
@@ -720,9 +721,6 @@ def _load_plugin_config_partial(plugin_id):
# for one installed as ledmatrix-<id> is not plugins_dir/<id>.
_plugin_dir = _resolved_plugin_dir(plugin_id) or _plugin_dir
# Get plugin instance (may be None if not loaded)
plugin_instance = pages_v3.plugin_manager.get_plugin(plugin_id)
# Get plugin configuration from config file
config = {}
if pages_v3.config_manager:
@@ -829,8 +827,6 @@ def _load_plugin_config_partial(plugin_id):
# Determine enabled status
enabled = config.get('enabled', True)
if plugin_instance:
enabled = plugin_instance.enabled
# Build plugin data for template
plugin_data = {
@@ -868,7 +864,10 @@ def _load_starlark_config_partial(app_id):
return '<div class="text-red-500 p-4">Invalid app ID</div>', 400
try:
starlark_plugin = pages_v3.plugin_manager.get_plugin('starlark-apps') if pages_v3.plugin_manager else None
# Always None: the web process runs no plugin code. See
# api_v3._get_starlark_plugin, the one seam for this.
from web_interface.blueprints.api_v3 import _get_starlark_plugin
starlark_plugin = _get_starlark_plugin()
if starlark_plugin and hasattr(starlark_plugin, 'apps'):
app = starlark_plugin.apps.get(app_id)
web_interface/static/v3/app.js
+24 -11
View File
@@ -27,8 +27,9 @@
* the display" banner; the floating live preview; aria-current on the nav;
* the mobile nav drawer's keyboard handling; header widget placement.
*
* Globals: showSaveResult, showRestartPending, dismissRestartPending,
* restartPendingNow, toggleFloatingPreview, applyFloatingPreviewSize,
* Globals: showSaveResult, showRestartPending, noteRestartRequired,
* dismissRestartPending, restartPendingNow, toggleFloatingPreview,
* applyFloatingPreviewSize,
* cycleFloatingPreviewSize, updateFloatingPreviewVisibility,
* previewPluginNow, updateNavAriaCurrent, placeHeaderWidgets.
*/
@@ -72,19 +73,31 @@ document.body.addEventListener('htmx:afterRequest', function(event) {
}
}
// Main-config saves (display hardware, rotation/durations, general) only
// take effect after a display-service restart — surface the reminder
// banner. Plugin config saves apply live and are deliberately excluded.
// A response that needs a display restart to take effect says so with
// restart_required (main-config saves, store operations the display
// cannot pick up live); surface the reminder banner for it.
try {
const cfg = event.detail.requestConfig;
if (cfg && cfg.verb === 'post' &&
(cfg.path || '').includes('/api/v3/config/main') &&
response && response.status >= 200 && response.status < 300) {
window.showRestartPending();
if (response && response.status >= 200 && response.status < 300 && response.responseText) {
window.noteRestartRequired(JSON.parse(response.responseText));
}
} catch { /* banner is best-effort */ }
} catch { /* not JSON; the banner is best-effort */ }
});
/**
* Shows the restart-pending banner when an API response says the change
* needs a display restart (`restart_required: true`), with the response's
* `restart_message` as its wording when there is one. Every caller of an
* endpoint that can answer this way passes the parsed body here, so the
* server alone decides when the banner appears.
* @param {Object} data - a parsed JSON response body (or an operation result)
* @returns {boolean} whether the banner was shown
*/
window.noteRestartRequired = function(data) {
if (!data || data.restart_required !== true) return false;
window.showRestartPending(typeof data.restart_message === 'string' ? data.restart_message : undefined);
return true;
};
/**
* Shows the outcome of a settings form save as one notification. Used by the
* hx-on:htmx:after-request of the Display, Rotation & Durations and General
web_interface/static/v3/js/app-shell.js
+2
View File
@@ -1366,6 +1366,8 @@ function markPanelLoadFailed(event) {
const data = await response.json();
showNotification(data.message, data.status);
// The display keeps running the old code until it restarts.
window.noteRestartRequired(data);
if (data.status === 'success') {
// Refresh the plugin list
web_interface/static/v3/js/plugins/install_manager.js
+55 -3
View File
@@ -69,10 +69,20 @@ const PluginInstallManager = {
if (onProgress) onProgress(i + 1, plugins.length, plugin.id);
// Each plugin gets its own pass over the backoff schedule.
const pendingDelays = retryDelays.slice();
let lostAnswer = false;
for (;;) {
try {
const result = await window.PluginAPI.updatePlugin(plugin.id);
results.push({ pluginId: plugin.id, success: true, result });
const entry = { pluginId: plugin.id, success: true, result };
if (lostAnswer) {
// An earlier attempt got no answer, so it may have
// updated the plugin before the connection dropped,
// and this answer then says up_to_date. restartRequest()
// reads these two.
entry.afterLostAnswer = true;
entry.enabled = plugin.enabled === true;
}
results.push(entry);
break;
} catch (error) {
// No HTTP answer at all (connection refused/reset, e.g. the
@@ -81,6 +91,7 @@ const PluginInstallManager = {
// back rather than skipping it. An HTTP error response is
// the server's answer and is not retried.
if (error && error.error_code === 'NETWORK_ERROR' && pendingDelays.length > 0) {
lostAnswer = true;
await sleep(pendingDelays.shift());
continue;
}
@@ -90,9 +101,14 @@ const PluginInstallManager = {
}
}
// Reload plugin list once at the end
// Reload plugin list once at the end. A failed refresh must not
// lose the results: they carry the restart flags.
if (window.PluginStateManager) {
await window.PluginStateManager.loadInstalledPlugins();
try {
await window.PluginStateManager.loadInstalledPlugins();
} catch (error) {
console.warn('Could not refresh the installed plugin list after updating:', error);
}
}
return results;
@@ -148,6 +164,42 @@ const PluginInstallManager = {
text: parts.join(', '),
type
};
},
/**
* The first update answer that says the display needs a restart, or null.
*
* The display keeps running the code it loaded until it restarts, so an
* update of a plugin it runs answers `restart_required: true` (with the
* banner's wording in `restart_message`). One restart covers every
* plugin in the run, so one answer is enough; pass it to
* window.noteRestartRequired.
*
* Failing that, an enabled plugin whose first request got no answer and
* whose re-sent one says up_to_date may have been updated by the lost
* request, which nothing reported: that asks for a restart too, since a
* needless restart is cheaper than the display running old code.
*
* @param {Array} results - updateAll()'s results
* @returns {Object|null}
*/
restartRequest(results) {
const entries = Array.isArray(results) ? results : [];
for (const entry of entries) {
const body = entry && entry.success ? entry.result : null;
if (body && body.restart_required === true) return body;
}
for (const entry of entries) {
if (entry && entry.success && entry.afterLostAnswer && entry.enabled
&& this.updateOutcome(entry) === 'up_to_date') {
return {
restart_required: true,
restart_message: `Plugin ${entry.pluginId} may have been updated before the `
+ 'connection dropped — restart the display to be sure it runs the new version',
};
}
}
return null;
}
};
web_interface/static/v3/plugins_manager.js
+11
View File
@@ -453,6 +453,7 @@ window.handleGitHubPluginInstall = function() {
urlInput.value = '';
showNotification(`Plugin ${data.plugin_id} installed successfully`, 'success');
window.noteRestartRequired(data);
setTimeout(() => window.pluginManager.loadInstalledPlugins(true).catch(() => {}), 1000);
} else {
@@ -1548,6 +1549,9 @@ function runUpdateAllPlugins() {
// a no-op update is "already up to date", not "updated".
const summary = window.PluginInstallManager.summarizeUpdateResults(results);
showNotification(summary.text, summary.type);
// An updated plugin the display is running keeps its old code
// until the display restarts.
window.noteRestartRequired(window.PluginInstallManager.restartRequest(results));
})
.catch(error => {
console.error('Error updating all plugins:', error);
@@ -2062,6 +2066,7 @@ window.uninstallPlugin = function(pluginId) {
pollOperationStatus(operationId, pluginId, pluginName);
} else if (data.status === 'success') {
// Direct uninstall completed immediately
window.noteRestartRequired(data);
handleUninstallSuccess(pluginId);
} else {
// Error response
@@ -2104,6 +2109,9 @@ function pollOperationStatus(operationId, pluginId, pluginName, options = {}) {
const status = operation.status;
if (status === 'completed') {
// The operation's result says whether the display picks
// the change up by itself or needs a restart.
window.noteRestartRequired(operation.result);
onComplete();
} else if (status === 'failed') {
onFailed(operation.error || operation.message);
@@ -2549,6 +2557,7 @@ window.installPlugin = function(pluginId, branch = null) {
});
} else {
// No operation queue configured - install already completed synchronously.
window.noteRestartRequired(data);
enableAfterInstall();
}
})
@@ -2579,6 +2588,7 @@ window.installFromCustomRegistry = function(pluginId, registryUrl, pluginPath, b
.then(data => {
if (data.status === 'success') {
showNotification(`Plugin ${data.plugin_id} installed successfully`, 'success');
window.noteRestartRequired(data);
// Refresh installed plugins and re-render custom registry
loadInstalledPlugins().catch(() => {});
// Re-render custom registry to update install buttons
@@ -2771,6 +2781,7 @@ function attachInstallButtonHandler() {
pluginStatusDiv.innerHTML = `<span class="text-green-600"><i class="fas fa-check-circle mr-1"></i>Successfully installed: ${escapeHtml(data.plugin_id)}</span>`;
}
pluginUrlInput.value = '';
window.noteRestartRequired(data);
// Refresh installed plugins list
setTimeout(() => {
web_interface/templates/v3/base.html
+5 -4
View File
@@ -456,10 +456,11 @@
</div>
</div>
<!-- Restart-pending banner: shown after a main-config save (display
hardware, rotation order, durations, general settings) — those only
take effect after the display service restarts. Plugin config saves
apply live and don't trigger this. Wired in app.js. -->
<!-- Restart-pending banner: shown when a response says restart_required
-- a main-config save (display hardware, rotation order, durations,
general settings), or a plugin install, update or uninstall the
running display cannot pick up live. Plugin config saves apply live
and don't trigger this. Wired in app.js (noteRestartRequired). -->
<div id="restart-pending-banner" style="display:none"
class="update-banner border-b transition-all duration-300 ease-in-out">
<div class="mx-auto px-4 sm:px-6 lg:px-8 xl:px-12 2xl:px-16 py-2" style="max-width:100%">