mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-04-10 13:02:59 +00:00
Feat/monorepo migration (#238)
* feat: adapt LEDMatrix for monorepo plugin architecture Update store_manager to fetch manifests from subdirectories within the monorepo (plugin_path/manifest.json) instead of repo root. Remove 21 plugin submodule entries from .gitmodules, simplify workspace file to reference the monorepo, and clean up scripts for the new layout. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: auto-reinstall plugins when registry repo URL changes When a user clicks "Update" on a git-cloned plugin, detect if the local git remote URL no longer matches the registry's repo URL (e.g. after monorepo migration). Instead of pulling from the stale archived repo, automatically remove and reinstall from the new registry source. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: plugin store "View" button links to correct monorepo subdirectory When a plugin has a plugin_path (monorepo plugin), construct the GitHub URL as repo/tree/main/plugin_path so users land on the specific plugin directory. Pass plugin_path through the store API response to the frontend. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: monorepo manifest fetch in search + version-based update detection Fix search_plugins() to pass plugin_path when fetching manifests from GitHub, matching the fix already in get_plugin_info(). Without this, monorepo plugin descriptions 404 in search results. Add version comparison for non-git plugins (monorepo installs) so "Update All" skips plugins already at latest_version instead of blindly reinstalling every time. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: show plugin version instead of misleading monorepo commit info Replace commit hash, date, and stars on plugin cards with the plugin's version number. In a monorepo all plugins share the same commit history and star count, making those fields identical and misleading. Version is the meaningful per-plugin signal users care about. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add CLAUDE.md with project structure and plugin store docs Documents plugin store architecture, monorepo install flow, version- based update detection, and the critical version bump workflow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * perf: extract only target plugin from monorepo ZIP instead of all files Previously _install_from_monorepo() called extractall() on the entire monorepo ZIP (~13MB, 600+ files) just to grab one plugin subdirectory. Now filter zip members by the plugin prefix and extract only matching files, reducing disk I/O by ~96% per install/update. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * perf: download only target plugin files via GitHub Trees API Replace full monorepo ZIP download (~5MB) with targeted file downloads (~200KB per plugin) using the GitHub Git Trees API for directory listing and raw.githubusercontent.com for individual file content. One API call fetches the repo tree, client filters for the target plugin's files, then downloads each file individually. Falls back to ZIP if the API is unavailable (rate limited, no network, etc.). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: clean up partial files between API and ZIP install fallbacks Ensure target_path is fully removed before the ZIP fallback runs, and before shutil.move() in the ZIP method. Prevents directory nesting if the API method creates target_path then fails mid-download. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: harden scripts and fix monorepo URL handling - setup_plugin_repos.py: add type hints, remove unnecessary f-string, wrap manifest parsing in try/except to skip malformed manifests - update_plugin_repos.py: add 120s timeout to git pull with TimeoutExpired handling - store_manager.py: fix rstrip('.zip') stripping valid branch chars, use removesuffix('.zip'); remove redundant import json - plugins_manager.js: View button uses dynamic branch, disables when repo is missing, encodes plugin_path in URL - CLAUDE.md: document plugin repo naming convention Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: harden monorepo install security and cleanup - store_manager: fix temp dir leak in _install_from_monorepo_zip by moving cleanup to finally block - store_manager: add zip-slip guard validating extracted paths stay inside temp directory - store_manager: add 500-file sanity cap to API-based install - store_manager: extract _normalize_repo_url as @staticmethod - setup_plugin_repos: propagate create_symlinks() failure via sys.exit, narrow except to OSError Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add path traversal guard to API-based monorepo installer Validate that each file's resolved destination stays inside target_path before creating directories or writing bytes, mirroring the zip-slip guard in _install_from_monorepo_zip. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use _safe_remove_directory for monorepo migration cleanup Replace shutil.rmtree(ignore_errors=True) with _safe_remove_directory which handles permission errors gracefully and returns status, preventing install_plugin from running against a partially-removed directory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Chuck
@@ -1,151 +1,89 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Setup plugin repository references for multi-root workspace.
|
||||
Setup plugin repository symlinks for local development.
|
||||
|
||||
This script creates symlinks in plugin-repos/ pointing to the actual
|
||||
plugin repositories in the parent directory, allowing the system to
|
||||
find plugins without modifying the LEDMatrix project structure.
|
||||
Creates symlinks in plugin-repos/ pointing to plugin directories
|
||||
in the ledmatrix-plugins monorepo.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Paths
|
||||
PROJECT_ROOT = Path(__file__).parent.parent
|
||||
PLUGIN_REPOS_DIR = PROJECT_ROOT / "plugin-repos"
|
||||
GITHUB_DIR = PROJECT_ROOT.parent
|
||||
CONFIG_FILE = PROJECT_ROOT / "config" / "config.json"
|
||||
MONOREPO_PLUGINS = PROJECT_ROOT.parent / "ledmatrix-plugins" / "plugins"
|
||||
|
||||
|
||||
def get_workspace_plugins():
|
||||
"""Get list of plugins from workspace file."""
|
||||
workspace_file = PROJECT_ROOT / "LEDMatrix.code-workspace"
|
||||
if not workspace_file.exists():
|
||||
return []
|
||||
|
||||
try:
|
||||
with open(workspace_file, 'r') as f:
|
||||
workspace = json.load(f)
|
||||
except json.JSONDecodeError as e:
|
||||
print(f"Error: Failed to parse workspace file {workspace_file}: {e}")
|
||||
print("Please check that the workspace file contains valid JSON.")
|
||||
return []
|
||||
|
||||
plugins = []
|
||||
for folder in workspace.get('folders', []):
|
||||
path = folder.get('path', '')
|
||||
if path.startswith('../') and path != '../ledmatrix-plugins':
|
||||
plugin_name = path.replace('../', '')
|
||||
plugins.append({
|
||||
'name': plugin_name,
|
||||
'workspace_path': path,
|
||||
'actual_path': GITHUB_DIR / plugin_name,
|
||||
'link_path': PLUGIN_REPOS_DIR / plugin_name
|
||||
})
|
||||
|
||||
return plugins
|
||||
def parse_json_with_trailing_commas(text: str) -> dict:
|
||||
"""Parse JSON that may have trailing commas."""
|
||||
text = re.sub(r",\s*([}\]])", r"\1", text)
|
||||
return json.loads(text)
|
||||
|
||||
|
||||
def create_symlinks():
|
||||
"""Create symlinks in plugin-repos/ pointing to actual repos."""
|
||||
plugins = get_workspace_plugins()
|
||||
|
||||
if not plugins:
|
||||
print("No plugins found in workspace configuration")
|
||||
def create_symlinks() -> bool:
|
||||
"""Create symlinks in plugin-repos/ pointing to monorepo plugin dirs."""
|
||||
if not MONOREPO_PLUGINS.exists():
|
||||
print(f"Error: Monorepo plugins directory not found: {MONOREPO_PLUGINS}")
|
||||
return False
|
||||
|
||||
# Ensure plugin-repos directory exists
|
||||
|
||||
PLUGIN_REPOS_DIR.mkdir(exist_ok=True)
|
||||
|
||||
|
||||
created = 0
|
||||
skipped = 0
|
||||
errors = 0
|
||||
|
||||
print(f"Setting up plugin repository links...")
|
||||
print(f" Source: {GITHUB_DIR}")
|
||||
|
||||
print("Setting up plugin symlinks...")
|
||||
print(f" Source: {MONOREPO_PLUGINS}")
|
||||
print(f" Links: {PLUGIN_REPOS_DIR}")
|
||||
print()
|
||||
|
||||
for plugin in plugins:
|
||||
actual_path = plugin['actual_path']
|
||||
link_path = plugin['link_path']
|
||||
|
||||
if not actual_path.exists():
|
||||
print(f" ⚠️ {plugin['name']} - source not found: {actual_path}")
|
||||
errors += 1
|
||||
|
||||
for plugin_dir in sorted(MONOREPO_PLUGINS.iterdir()):
|
||||
if not plugin_dir.is_dir():
|
||||
continue
|
||||
|
||||
# Remove existing link/file if it exists
|
||||
manifest_path = plugin_dir / "manifest.json"
|
||||
if not manifest_path.exists():
|
||||
continue
|
||||
|
||||
try:
|
||||
with open(manifest_path, "r", encoding="utf-8") as f:
|
||||
manifest = parse_json_with_trailing_commas(f.read())
|
||||
except (OSError, json.JSONDecodeError) as e:
|
||||
print(f" {plugin_dir.name} - failed to read {manifest_path}: {e}")
|
||||
continue
|
||||
plugin_id = manifest.get("id", plugin_dir.name)
|
||||
link_path = PLUGIN_REPOS_DIR / plugin_id
|
||||
|
||||
if link_path.exists() or link_path.is_symlink():
|
||||
if link_path.is_symlink():
|
||||
# Check if it points to the right place
|
||||
try:
|
||||
if link_path.resolve() == actual_path.resolve():
|
||||
print(f" ✓ {plugin['name']} - link already exists")
|
||||
if link_path.resolve() == plugin_dir.resolve():
|
||||
skipped += 1
|
||||
continue
|
||||
else:
|
||||
# Remove old symlink pointing elsewhere
|
||||
link_path.unlink()
|
||||
except Exception as e:
|
||||
print(f" ⚠️ {plugin['name']} - error checking link: {e}")
|
||||
except OSError:
|
||||
link_path.unlink()
|
||||
else:
|
||||
# It's a directory/file, not a symlink
|
||||
print(f" ⚠️ {plugin['name']} - {link_path.name} exists but is not a symlink")
|
||||
print(f" Skipping (manual cleanup required)")
|
||||
print(f" {plugin_id} - exists but is not a symlink, skipping")
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
# Create symlink
|
||||
try:
|
||||
# Use relative path for symlink portability
|
||||
relative_path = os.path.relpath(actual_path, link_path.parent)
|
||||
link_path.symlink_to(relative_path)
|
||||
print(f" ✓ {plugin['name']} - linked")
|
||||
created += 1
|
||||
except Exception as e:
|
||||
print(f" ✗ {plugin['name']} - failed to create link: {e}")
|
||||
errors += 1
|
||||
|
||||
print()
|
||||
print(f"✅ Created {created} links, skipped {skipped}, errors {errors}")
|
||||
|
||||
return errors == 0
|
||||
|
||||
relative_path = os.path.relpath(plugin_dir, link_path.parent)
|
||||
link_path.symlink_to(relative_path)
|
||||
print(f" {plugin_id} - linked")
|
||||
created += 1
|
||||
|
||||
def update_config_path():
|
||||
"""Update config to use absolute path to parent directory (alternative approach)."""
|
||||
# This is an alternative - set plugins_directory to absolute path
|
||||
# Currently not implemented as symlinks are preferred
|
||||
pass
|
||||
print(f"\nCreated {created} links, skipped {skipped}")
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
"""Main function."""
|
||||
print("🔗 Setting up plugin repository symlinks...")
|
||||
print()
|
||||
|
||||
if not GITHUB_DIR.exists():
|
||||
print(f"Error: GitHub directory not found: {GITHUB_DIR}")
|
||||
return 1
|
||||
|
||||
success = create_symlinks()
|
||||
|
||||
if success:
|
||||
print()
|
||||
print("✅ Plugin repository setup complete!")
|
||||
print()
|
||||
print("Plugins are now accessible via symlinks in plugin-repos/")
|
||||
print("You can update plugins independently in their git repos.")
|
||||
return 0
|
||||
else:
|
||||
print()
|
||||
print("⚠️ Setup completed with some errors. Check output above.")
|
||||
return 1
|
||||
print("Setting up plugin repository symlinks from monorepo...\n")
|
||||
if not create_symlinks():
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
Reference in New Issue
Block a user