Files
LEDMatrix/test/test_saved_repositories.py
T
ChuckandClaude Opus 5.5 b11bcfa204 fix(plugins): store and plugin-manager bugs; tidy src/plugin_system (#635)
* fix(store): don't read a ZIP-installed plugin's remote from the LEDMatrix repo

update_plugin looked up remote.origin.url with `git -C <plugin> config
--local` for plugins that are not git checkouts. Under plugin-repos/ git
walks up to the enclosing LEDMatrix repository, so the lookup returned
LEDMatrix's own URL and a plugin missing from the registry was
"reinstalled" from the LEDMatrix repo. Only ask git when the plugin
directory has its own .git, the test _get_local_git_info already uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(schema): report each missing required field once, by name

validate_config_against_schema ran its own required-fields loop after
Draft7Validator.iter_errors, which already yields one `required` error
per missing field, so every missing top-level field was listed twice.
The validator's copy also printed the schema's whole `required` list
("Missing required property '['api_key', 'city']'") instead of the field.
Drop the loop and take the field name from the error itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(store): stop mangling repository URLs that contain ".git"

install_from_url and fetch_registry_from_url cleaned URLs with
`rstrip('/').replace('.git', '')`, which removes ".git" anywhere:
https://github.com/user/my.github.io became .../myhub.io, so installing
or browsing that repository asked GitHub for one that does not exist.

Add src/plugin_system/repo_urls.py with one anchored normalize_repo_url(),
same_repo() for comparisons, github_owner_repo() and github_api_headers(),
and use them for the five copies of the owner/repo parsing and GitHub
headers in the store and for saved repositories. GitHub URLs are now
recognised by urlparse().hostname everywhere: _get_latest_commit_info
used a substring test, and _install_from_monorepo_api parsed any host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(store): install a repository whose only branch is not main/master

_install_via_git returned None both when every clone failed and when the
last-resort clone of the repository's default branch succeeded.
_install_plugin_impl papered over it with `and not plugin_path.exists()`;
install_from_url did not, so a repository whose only branch is e.g.
`develop` was cloned, then treated as a failure, then "downloaded" from
main/master archives that do not exist.

After a default-branch clone, return the branch the clone checked out
(read from .git/HEAD), so None means failure and nothing else, and give
both callers the same `branch_used is None` fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): judge the memory limit on each call's own growth

monitor_call stores `metrics.memory_mb = max(previous, growth)`, and
_check_limits compared that high-water mark with max_memory_mb. It never
decreases, so once one update() grew the process past the limit every
later call raised ResourceLimitExceeded and the circuit breaker kept
reopening. Pass the call's own RSS growth to _check_limits; keep the
high-water mark for reporting and document what it measures.

Remove ResourceMetrics.update_average_execution_time: nothing called it,
and it overwrote the running total with the average.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): reload_plugin re-reads the manifest from the discovered directory

reload_plugin read `plugins_dir / plugin_id / "manifest.json"`, ignoring
the discovery map and the plugin_dirs rules. For a plugin whose
directory name differs from its manifest id the path did not exist, the
re-read was skipped without a word, and the reload kept the stale
manifest. Resolve the directory with find_plugin_directory, as
load_plugin does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): drop the always-null last_display from plugin state info

PluginStateManager reported `last_display` from `_last_display`, which
nothing ever wrote, so it was null for every plugin. Recording it in
PluginExecutor.execute_display would not help: get_state_info's only
reader is the web process, whose PluginManager never calls display().
Remove the field, its dict and get_last_display() (no caller in core,
the web UI or the plugin monorepo).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(store): share the rollback and requirements helpers, drop dead code

- install_plugin and _reinstall_with_rollback set aside, discard and
  restore the old copy through _set_aside/_discard_backup/_restore_backup
  instead of two copies of the same blocks.
- The loader and the store run the same pre-pip checks through
  contained_plugin_dir() and requirements_to_install() in plugin_loader.
  They still invoke pip differently (sys.executable -m pip vs. the sudo
  wrapper). `except (BrokenPipeError, OSError)` + `isinstance(e, OSError)`
  becomes `except OSError` checking errno.EPIPE.
- load_module never returns None, so load_plugin's check is gone and the
  docstring says what it raises.
- Remove the always-true JSONSCHEMA_AVAILABLE, the inline re-imports of
  re and permission_utils, the fake status_result object nobody reads,
  hasattr(git_error, 'cmd'), a redundant "merge conflict" test and
  `import traceback` (exc_info=True does it).
- Correct comments: install_from_url names the directory for the
  caller's id when given (not always the manifest id), _get_local_git_info
  saves one git subprocess (not four), _enrich calls two helpers,
  search_plugins documents all its arguments, _find_plugin_path states
  its behaviour instead of a TODO, and history narration is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(plugins): tidy base_plugin, correct plugin_manager/state comments

- base_plugin: drop the unused `import logging`; get_display_duration
  runs the instance value and the config value through one
  _positive_seconds() helper instead of two copies of the coercion; the
  'static'/'none'/fallback branches of get_vegas_display_mode, which all
  returned FIXED_SEGMENT, are one; fix the mis-indented validate_config
  example; say that get_supported_vegas_modes/get_vegas_segment_width
  are not consulted by core (kept, plugins override them).
- schema_manager: import expand_style_elements normally rather than
  swallowing an ImportError of a core module.
- plugin_manager: the plugins directory is the configured one
  (plugin-repos/ by default), not plugins/; get_config() returns the live
  dict, not a copy, so the interval cache comments say what it saves.
- state_manager: config_version and the file version are not used to
  detect corruption; say what they are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(plugins): stop writing data/plugin_operations.json

PluginOperationQueue wrote its finished-operation history to
data/plugin_operations.json after every operation, and read it back only
into its own in-memory list, which only get_operation_history() exposes
-- and nothing calls that. The operation-history endpoint reads
OperationHistory (data/operation_history.json). No code in src/,
web_interface/, scripts/ or test/ reads the file.

Drop the history_file/lazy_load parameters and the load/save code; the
bounded in-memory history stays. web_interface/app.py and the
integration test stop passing the removed arguments. An existing
data/plugin_operations.json is left in place (data/* is gitignored).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): plugin-system

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 17:32:02 -04:00

241 lines
10 KiB
Python

"""
Tests for src/plugin_system/saved_repositories.py — pins the
SavedRepositoriesManager contract.
Covers: the three accepted on-disk load shapes (bare list, wrapped
{"repositories": [...]}, anything else -> []) and that saves always write
the bare-list form; add/remove/has round trips through a fresh manager;
URL normalization post-fix (normalize_repo_url strips only a TRAILING '.git' after
trailing slashes — the old unanchored .replace('.git', '') mangled URLs
like my.github.io); name derivation and registry-vs-single type
classification (the ledmatrix-plugins check is lowercased, the
plugins.json check is case-sensitive); and the post-fix rollback of the
in-memory list when _save_repositories() fails, so memory never diverges
from disk.
"""
import json
from src.plugin_system.saved_repositories import SavedRepositoriesManager
def make_manager(path):
return SavedRepositoriesManager(config_path=str(path))
class TestLoading:
def test_missing_file_empty_and_not_created(self, tmp_path):
path = tmp_path / "repos.json"
manager = make_manager(path)
assert manager.get_all() == []
assert not path.exists()
def test_bare_list_shape(self, tmp_path):
path = tmp_path / "repos.json"
entries = [{'url': 'https://github.com/u/r', 'name': 'r', 'type': 'single'}]
path.write_text(json.dumps(entries))
assert make_manager(path).get_all() == entries
def test_wrapped_dict_shape(self, tmp_path):
path = tmp_path / "repos.json"
entries = [{'url': 'https://github.com/u/r', 'name': 'r', 'type': 'single'}]
path.write_text(json.dumps({'repositories': entries}))
assert make_manager(path).get_all() == entries
def test_other_shape_yields_empty(self, tmp_path):
path = tmp_path / "repos.json"
path.write_text(json.dumps({'x': 1}))
assert make_manager(path).get_all() == []
def test_malformed_json_yields_empty_no_raise(self, tmp_path):
path = tmp_path / "repos.json"
path.write_text("not json {{")
assert make_manager(path).get_all() == []
class TestSaveFormat:
def test_save_always_writes_bare_list(self, tmp_path):
# Even when loaded from the wrapped {"repositories": [...]} form,
# the next save normalizes the file to a bare JSON list.
path = tmp_path / "repos.json"
entries = [{'url': 'https://github.com/u/r', 'name': 'r', 'type': 'single'}]
path.write_text(json.dumps({'repositories': entries}))
manager = make_manager(path)
assert manager.add("https://github.com/u/r2") is True
on_disk = json.loads(path.read_text())
assert isinstance(on_disk, list)
assert len(on_disk) == 2
class TestAdd:
def test_round_trip_creates_parents_and_reloads(self, tmp_path):
path = tmp_path / "sub" / "repos.json"
manager = make_manager(path)
assert manager.add("https://github.com/user/repo") is True
assert path.exists()
entry = manager.get_all()[0]
assert entry['url'] == "https://github.com/user/repo"
assert entry['name'] == "repo"
assert entry['type'] == "single"
# A fresh manager on the same path sees the persisted entry.
fresh = make_manager(path)
assert fresh.get_all() == [entry]
def test_duplicate_returns_false_file_unchanged(self, tmp_path):
path = tmp_path / "repos.json"
manager = make_manager(path)
assert manager.add("https://github.com/user/repo") is True
before = path.read_text()
assert manager.add("https://github.com/user/repo") is False
assert path.read_text() == before
assert len(manager.get_all()) == 1
def test_trailing_git_and_slash_stripped(self, tmp_path):
manager = make_manager(tmp_path / "repos.json")
assert manager.add("https://github.com/user/repo.git/") is True
assert manager.get_all()[0]['url'] == "https://github.com/user/repo"
def test_interior_dot_git_not_mangled(self, tmp_path):
# Regression for the old unanchored .replace('.git', ''): a URL
# merely CONTAINING '.git' must be stored verbatim.
manager = make_manager(tmp_path / "repos.json")
url = "https://github.com/user/my.github.io"
assert manager.add(url) is True
assert manager.get_all()[0]['url'] == url
class TestNameExtraction:
def test_name_derived_from_last_path_segment(self, tmp_path):
manager = make_manager(tmp_path / "repos.json")
manager.add("https://github.com/user/football-scoreboard")
assert manager.get_all()[0]['name'] == "football-scoreboard"
def test_explicit_name_preserved(self, tmp_path):
manager = make_manager(tmp_path / "repos.json")
manager.add("https://github.com/user/repo", name="My Repo")
assert manager.get_all()[0]['name'] == "My Repo"
def test_url_without_slash_uses_whole_url(self, tmp_path):
manager = make_manager(tmp_path / "repos.json")
manager.add("standalone")
assert manager.get_all()[0]['name'] == "standalone"
class TestTypeClassification:
def _type_of(self, tmp_path, url):
manager = make_manager(tmp_path / "repos.json")
assert manager.add(url) is True
return manager.get_all()[0]['type']
def test_plugins_json_url_is_registry(self, tmp_path):
url = "https://raw.githubusercontent.com/x/main/plugins.json"
assert self._type_of(tmp_path, url) == "registry"
def test_ledmatrix_plugins_check_is_case_insensitive(self, tmp_path):
url = "https://github.com/ChuckBuilds/LEDMATRIX-PLUGINS"
assert self._type_of(tmp_path, url) == "registry"
def test_plugins_json_check_is_case_sensitive(self, tmp_path):
# Only the 'ledmatrix-plugins' check is lowercased; the
# 'plugins.json' substring check is case-sensitive. Pinned.
url = "https://example.com/PLUGINS.JSON"
assert self._type_of(tmp_path, url) == "single"
def test_plain_repo_is_single(self, tmp_path):
assert self._type_of(tmp_path, "https://github.com/user/repo") == "single"
def test_get_registry_repositories_filters(self, tmp_path):
manager = make_manager(tmp_path / "repos.json")
manager.add("https://github.com/user/repo")
manager.add("https://raw.githubusercontent.com/x/main/plugins.json")
registries = manager.get_registry_repositories()
assert len(registries) == 1
assert registries[0]['type'] == "registry"
class TestRemove:
def test_remove_present_persists(self, tmp_path):
path = tmp_path / "repos.json"
manager = make_manager(path)
manager.add("https://github.com/user/repo")
assert manager.remove("https://github.com/user/repo") is True
assert manager.get_all() == []
assert make_manager(path).get_all() == []
def test_remove_absent_false_no_write(self, tmp_path):
path = tmp_path / "repos.json"
manager = make_manager(path)
manager.add("https://github.com/user/repo")
before = path.read_text()
assert manager.remove("https://github.com/user/other") is False
assert path.read_text() == before
def test_remove_with_dirty_url_matches_clean_stored(self, tmp_path):
manager = make_manager(tmp_path / "repos.json")
manager.add("https://github.com/user/repo")
assert manager.remove("https://github.com/user/repo.git/") is True
assert manager.get_all() == []
class TestHas:
def test_has_applies_url_cleaning(self, tmp_path):
manager = make_manager(tmp_path / "repos.json")
manager.add("https://x/y")
assert manager.has("https://x/y.git/") is True
assert manager.has("https://x/z") is False
class TestSaveFailureRollback:
def test_add_rolls_back_on_save_failure(self, tmp_path, monkeypatch):
# Post-fix: a failed save must not leave a phantom in-memory entry.
path = tmp_path / "repos.json"
manager = make_manager(path)
monkeypatch.setattr(manager, "_save_repositories", lambda: False)
assert manager.add("https://github.com/user/repo") is False
assert manager.get_all() == []
assert not path.exists()
def test_remove_rolls_back_on_save_failure(self, tmp_path, monkeypatch):
path = tmp_path / "repos.json"
manager = make_manager(path)
manager.add("https://github.com/user/repo") # real save
monkeypatch.setattr(manager, "_save_repositories", lambda: False)
assert manager.remove("https://github.com/user/repo") is False
assert manager.get_all() == [
{'url': 'https://github.com/user/repo', 'name': 'repo', 'type': 'single'}
]
# Disk still has the entry too — memory and disk stay in sync.
assert len(json.loads(path.read_text())) == 1
def test_failed_write_leaves_existing_file_intact(self, tmp_path, monkeypatch):
# The save is atomic (temp file + os.replace): a write that dies
# mid-serialization must neither truncate the existing file nor
# leave a stray .tmp behind.
path = tmp_path / "repos.json"
manager = make_manager(path)
manager.add("https://github.com/user/repo") # real save
before = path.read_text()
def boom(*args, **kwargs):
raise OSError("disk full")
monkeypatch.setattr(json, "dump", boom)
assert manager.add("https://github.com/user/other") is False
assert path.read_text() == before
assert list(tmp_path.glob("*.tmp")) == []
class TestGetAllCopy:
def test_get_all_is_shallow_copy(self, tmp_path):
# Characterization: get_all() copies the LIST but not the entry
# dicts, so mutating a returned entry mutates internal state.
# Appending to the returned list, however, does not. Do not "fix"
# without auditing callers that rely on list-copy semantics.
manager = make_manager(tmp_path / "repos.json")
manager.add("https://github.com/user/repo")
returned = manager.get_all()
returned.append({'url': 'x'})
assert len(manager.get_all()) == 1 # list itself is copied
manager.get_all()[0]['name'] = 'hacked'
assert manager.get_all()[0]['name'] == 'hacked' # dicts are shared