rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-04 14:25:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(plugins): store and plugin-manager bugs; tidy src/plugin_system (#635)

Browse Source 
* fix(store): don't read a ZIP-installed plugin's remote from the LEDMatrix repo

update_plugin looked up remote.origin.url with `git -C <plugin> config
--local` for plugins that are not git checkouts. Under plugin-repos/ git
walks up to the enclosing LEDMatrix repository, so the lookup returned
LEDMatrix's own URL and a plugin missing from the registry was
"reinstalled" from the LEDMatrix repo. Only ask git when the plugin
directory has its own .git, the test _get_local_git_info already uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(schema): report each missing required field once, by name

validate_config_against_schema ran its own required-fields loop after
Draft7Validator.iter_errors, which already yields one `required` error
per missing field, so every missing top-level field was listed twice.
The validator's copy also printed the schema's whole `required` list
("Missing required property '['api_key', 'city']'") instead of the field.
Drop the loop and take the field name from the error itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(store): stop mangling repository URLs that contain ".git"

install_from_url and fetch_registry_from_url cleaned URLs with
`rstrip('/').replace('.git', '')`, which removes ".git" anywhere:
https://github.com/user/my.github.io became .../myhub.io, so installing
or browsing that repository asked GitHub for one that does not exist.

Add src/plugin_system/repo_urls.py with one anchored normalize_repo_url(),
same_repo() for comparisons, github_owner_repo() and github_api_headers(),
and use them for the five copies of the owner/repo parsing and GitHub
headers in the store and for saved repositories. GitHub URLs are now
recognised by urlparse().hostname everywhere: _get_latest_commit_info
used a substring test, and _install_from_monorepo_api parsed any host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(store): install a repository whose only branch is not main/master

_install_via_git returned None both when every clone failed and when the
last-resort clone of the repository's default branch succeeded.
_install_plugin_impl papered over it with `and not plugin_path.exists()`;
install_from_url did not, so a repository whose only branch is e.g.
`develop` was cloned, then treated as a failure, then "downloaded" from
main/master archives that do not exist.

After a default-branch clone, return the branch the clone checked out
(read from .git/HEAD), so None means failure and nothing else, and give
both callers the same `branch_used is None` fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): judge the memory limit on each call's own growth

monitor_call stores `metrics.memory_mb = max(previous, growth)`, and
_check_limits compared that high-water mark with max_memory_mb. It never
decreases, so once one update() grew the process past the limit every
later call raised ResourceLimitExceeded and the circuit breaker kept
reopening. Pass the call's own RSS growth to _check_limits; keep the
high-water mark for reporting and document what it measures.

Remove ResourceMetrics.update_average_execution_time: nothing called it,
and it overwrote the running total with the average.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): reload_plugin re-reads the manifest from the discovered directory

reload_plugin read `plugins_dir / plugin_id / "manifest.json"`, ignoring
the discovery map and the plugin_dirs rules. For a plugin whose
directory name differs from its manifest id the path did not exist, the
re-read was skipped without a word, and the reload kept the stale
manifest. Resolve the directory with find_plugin_directory, as
load_plugin does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): drop the always-null last_display from plugin state info

PluginStateManager reported `last_display` from `_last_display`, which
nothing ever wrote, so it was null for every plugin. Recording it in
PluginExecutor.execute_display would not help: get_state_info's only
reader is the web process, whose PluginManager never calls display().
Remove the field, its dict and get_last_display() (no caller in core,
the web UI or the plugin monorepo).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(store): share the rollback and requirements helpers, drop dead code

- install_plugin and _reinstall_with_rollback set aside, discard and
  restore the old copy through _set_aside/_discard_backup/_restore_backup
  instead of two copies of the same blocks.
- The loader and the store run the same pre-pip checks through
  contained_plugin_dir() and requirements_to_install() in plugin_loader.
  They still invoke pip differently (sys.executable -m pip vs. the sudo
  wrapper). `except (BrokenPipeError, OSError)` + `isinstance(e, OSError)`
  becomes `except OSError` checking errno.EPIPE.
- load_module never returns None, so load_plugin's check is gone and the
  docstring says what it raises.
- Remove the always-true JSONSCHEMA_AVAILABLE, the inline re-imports of
  re and permission_utils, the fake status_result object nobody reads,
  hasattr(git_error, 'cmd'), a redundant "merge conflict" test and
  `import traceback` (exc_info=True does it).
- Correct comments: install_from_url names the directory for the
  caller's id when given (not always the manifest id), _get_local_git_info
  saves one git subprocess (not four), _enrich calls two helpers,
  search_plugins documents all its arguments, _find_plugin_path states
  its behaviour instead of a TODO, and history narration is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(plugins): tidy base_plugin, correct plugin_manager/state comments

- base_plugin: drop the unused `import logging`; get_display_duration
  runs the instance value and the config value through one
  _positive_seconds() helper instead of two copies of the coercion; the
  'static'/'none'/fallback branches of get_vegas_display_mode, which all
  returned FIXED_SEGMENT, are one; fix the mis-indented validate_config
  example; say that get_supported_vegas_modes/get_vegas_segment_width
  are not consulted by core (kept, plugins override them).
- schema_manager: import expand_style_elements normally rather than
  swallowing an ImportError of a core module.
- plugin_manager: the plugins directory is the configured one
  (plugin-repos/ by default), not plugins/; get_config() returns the live
  dict, not a copy, so the interval cache comments say what it saves.
- state_manager: config_version and the file version are not used to
  detect corruption; say what they are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(plugins): stop writing data/plugin_operations.json

PluginOperationQueue wrote its finished-operation history to
data/plugin_operations.json after every operation, and read it back only
into its own in-memory list, which only get_operation_history() exposes
-- and nothing calls that. The operation-history endpoint reads
OperationHistory (data/operation_history.json). No code in src/,
web_interface/, scripts/ or test/ reads the file.

Drop the history_file/lazy_load parameters and the load/save code; the
bounded in-memory history stays. web_interface/app.py and the
integration test stop passing the removed arguments. An existing
data/plugin_operations.json is left in place (data/* is gitignored).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): plugin-system

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-24 17:32:02 -04:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 3967a6cffc
commit b11bcfa204
24 changed files with 958 additions and 756 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/plugin_system/plugin_loader.py
+59 -43
View File
@@ -5,6 +5,7 @@ Handles plugin module imports, dependency installation, and class instantiation.
Extracted from PluginManager to improve separation of concerns.
"""
import errno
import importlib
import importlib.metadata
import importlib.util
@@ -184,6 +185,46 @@ def find_trusted_subdir(trusted_dir: str, name: str) -> Optional[str]:
return None
def contained_plugin_dir(plugin_dir: Path, plugins_dir: Path) -> Optional[str]:
"""``plugin_dir`` rebuilt from an entry enumerated under ``plugins_dir``.
Returns None when ``plugin_dir`` is not a subdirectory of ``plugins_dir``.
Callers derive ``plugin_dir`` from a manifest-declared id, so the path is
rebuilt from :func:`find_trusted_subdir`'s answer rather than trusted: a
name that came out of ``os.scandir()`` on the trusted root carries no
taint, which is a real containment guarantee (and one CodeQL's
path-injection query can follow), not a string sanitiser.
"""
plugin_dir_real = os.path.realpath(str(plugin_dir))
plugins_dir_real = os.path.realpath(str(plugins_dir))
matched_name = find_trusted_subdir(plugins_dir_real, os.path.basename(plugin_dir_real))
if matched_name is None:
return None
return os.path.join(plugins_dir_real, matched_name)
def requirements_to_install(plugin_dir: str, logger: logging.Logger,
label: str) -> Optional[str]:
"""The plugin's requirements.txt if pip has work to do, else None.
None when there is no requirements.txt, when it lists nothing (plugins
whose dependencies ship with core often keep an all-comments file), or
when every requirement is already installed. Shared by the loader and the
store so both skip pip for the same reasons; they differ only in how they
run it.
"""
requirements_file = os.path.join(plugin_dir, "requirements.txt")
if not os.path.isfile(requirements_file):
return None
if not requirements_has_real_deps(requirements_file):
logger.debug("requirements.txt for %s has no real dependencies, skipping pip", label)
return None
if requirements_are_satisfied(requirements_file):
logger.debug("Dependencies for %s already satisfied, skipping pip", label)
return None
return requirements_file
class PluginLoader:
"""Handles plugin module loading and class instantiation."""
@@ -273,43 +314,15 @@ class PluginLoader:
if not plugin_id:
return False
# Resolve to a canonical absolute path (normalises .. and symlinks)
plugin_dir_real = os.path.realpath(str(plugin_dir))
plugins_dir_real = os.path.realpath(str(plugins_dir))
requested_name = os.path.basename(plugin_dir_real)
# Match the requested directory against an entry actually enumerated
# from the trusted plugins_dir, and build the path from that entry --
# not from requested_name. A name that came out of os.scandir() on a
# trusted root carries no taint regardless of what the caller asked
# for, so this is a real containment guarantee (an allowlist check
# against a trusted source), not a string-sanitisation of untrusted
# input that a static analyzer has to trust blindly.
matched_name = find_trusted_subdir(plugins_dir_real, requested_name)
if matched_name is None:
safe_plugin_dir = contained_plugin_dir(plugin_dir, plugins_dir)
if safe_plugin_dir is None:
self.logger.error(
"Plugin directory for %s not found inside plugins dir", plugin_id
)
return False
safe_plugin_dir = os.path.join(plugins_dir_real, matched_name)
requirements_file = os.path.join(safe_plugin_dir, "requirements.txt")
if not os.path.isfile(requirements_file):
return True # No dependencies needed
if not requirements_has_real_deps(requirements_file):
self.logger.debug(
"requirements.txt for %s has no real dependencies (comments/blank only), skipping pip",
plugin_id
)
return True
if requirements_are_satisfied(requirements_file):
self.logger.debug(
"Dependencies for %s already satisfied in current environment, skipping pip",
plugin_id
)
requirements_file = requirements_to_install(safe_plugin_dir, self.logger, plugin_id)
if requirements_file is None:
return True
try:
@@ -348,8 +361,8 @@ class PluginLoader:
# below).
try:
# sys.executable is this process's own interpreter (not
# attacker-influenced), and requirements_file is a path
# built internally by find_plugin_directory, never raw
# attacker-influenced), and requirements_file is rebuilt
# by contained_plugin_dir() from a trusted listing, never raw
# external input.
retry_result = subprocess.run( # nosec B603 - no shell invoked (list-form argv) # nosemgrep
[sys.executable, "-m", "pip", "install", "--break-system-packages",
@@ -384,10 +397,10 @@ class PluginLoader:
except FileNotFoundError:
self.logger.warning("pip not found. Skipping dependency installation for %s", plugin_id)
return True
except (BrokenPipeError, OSError) as e:
# Handle broken pipe errors (errno 32) which can occur during pip downloads
# Often caused by network interruptions or output buffer issues
if isinstance(e, OSError) and e.errno == 32:
except OSError as e:
# A broken pipe (EPIPE) happens when pip's output pipe closes
# mid-download, usually a network interruption.
if e.errno == errno.EPIPE:
self.logger.error(
"Broken pipe error during dependency installation for %s. "
"This usually indicates a network interruption or pip output buffer issue. "
@@ -528,7 +541,7 @@ class PluginLoader:
plugin_id: str,
plugin_dir: Path,
entry_point: str
) -> Optional[Any]:
) -> Any:
"""
Load a plugin module from file.
@@ -547,7 +560,12 @@ class PluginLoader:
entry_point: Entry point filename (e.g., 'manager.py')
Returns:
Loaded module or None on error
The loaded module
Raises:
PluginError: If the plugin id, directory or entry point is
invalid. Whatever the module raises while executing
propagates unchanged.
"""
plugin_id = os.path.basename(plugin_id or '')
if not plugin_id:
@@ -782,9 +800,7 @@ class PluginLoader:
# Load module
entry_point = manifest.get('entry_point', 'manager.py')
module = self.load_module(plugin_id, plugin_dir, entry_point)
if module is None:
raise PluginError(f"Failed to load module for plugin {plugin_id}", plugin_id=plugin_id)
# Get plugin class
class_name = manifest.get('class_name')
if not class_name: