Feat/monorepo migration (#238)

* feat: adapt LEDMatrix for monorepo plugin architecture

Update store_manager to fetch manifests from subdirectories within the
monorepo (plugin_path/manifest.json) instead of repo root. Remove 21
plugin submodule entries from .gitmodules, simplify workspace file to
reference the monorepo, and clean up scripts for the new layout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: auto-reinstall plugins when registry repo URL changes

When a user clicks "Update" on a git-cloned plugin, detect if the
local git remote URL no longer matches the registry's repo URL (e.g.
after monorepo migration). Instead of pulling from the stale archived
repo, automatically remove and reinstall from the new registry source.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: plugin store "View" button links to correct monorepo subdirectory

When a plugin has a plugin_path (monorepo plugin), construct the GitHub
URL as repo/tree/main/plugin_path so users land on the specific plugin
directory. Pass plugin_path through the store API response to the
frontend.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: monorepo manifest fetch in search + version-based update detection

Fix search_plugins() to pass plugin_path when fetching manifests from
GitHub, matching the fix already in get_plugin_info(). Without this,
monorepo plugin descriptions 404 in search results.

Add version comparison for non-git plugins (monorepo installs) so
"Update All" skips plugins already at latest_version instead of blindly
reinstalling every time.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: show plugin version instead of misleading monorepo commit info

Replace commit hash, date, and stars on plugin cards with the plugin's
version number. In a monorepo all plugins share the same commit history
and star count, making those fields identical and misleading. Version
is the meaningful per-plugin signal users care about.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add CLAUDE.md with project structure and plugin store docs

Documents plugin store architecture, monorepo install flow, version-
based update detection, and the critical version bump workflow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf: extract only target plugin from monorepo ZIP instead of all files

Previously _install_from_monorepo() called extractall() on the entire
monorepo ZIP (~13MB, 600+ files) just to grab one plugin subdirectory.
Now filter zip members by the plugin prefix and extract only matching
files, reducing disk I/O by ~96% per install/update.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf: download only target plugin files via GitHub Trees API

Replace full monorepo ZIP download (~5MB) with targeted file downloads
(~200KB per plugin) using the GitHub Git Trees API for directory listing
and raw.githubusercontent.com for individual file content.

One API call fetches the repo tree, client filters for the target
plugin's files, then downloads each file individually. Falls back to
ZIP if the API is unavailable (rate limited, no network, etc.).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: clean up partial files between API and ZIP install fallbacks

Ensure target_path is fully removed before the ZIP fallback runs, and
before shutil.move() in the ZIP method. Prevents directory nesting if
the API method creates target_path then fails mid-download.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: harden scripts and fix monorepo URL handling

- setup_plugin_repos.py: add type hints, remove unnecessary f-string,
  wrap manifest parsing in try/except to skip malformed manifests
- update_plugin_repos.py: add 120s timeout to git pull with
  TimeoutExpired handling
- store_manager.py: fix rstrip('.zip') stripping valid branch chars,
  use removesuffix('.zip'); remove redundant import json
- plugins_manager.js: View button uses dynamic branch, disables when
  repo is missing, encodes plugin_path in URL
- CLAUDE.md: document plugin repo naming convention

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: harden monorepo install security and cleanup

- store_manager: fix temp dir leak in _install_from_monorepo_zip by
  moving cleanup to finally block
- store_manager: add zip-slip guard validating extracted paths stay
  inside temp directory
- store_manager: add 500-file sanity cap to API-based install
- store_manager: extract _normalize_repo_url as @staticmethod
- setup_plugin_repos: propagate create_symlinks() failure via sys.exit,
  narrow except to OSError

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add path traversal guard to API-based monorepo installer

Validate that each file's resolved destination stays inside
target_path before creating directories or writing bytes, mirroring
the zip-slip guard in _install_from_monorepo_zip.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use _safe_remove_directory for monorepo migration cleanup

Replace shutil.rmtree(ignore_errors=True) with _safe_remove_directory
which handles permission errors gracefully and returns status, preventing
install_plugin from running against a partially-removed directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-02-11 18:57:30 -05:00
committed by GitHub
parent 448a15c1e6
commit df3cf9bb56
9 changed files with 392 additions and 783 deletions

63
.gitmodules vendored
View File

@@ -1,66 +1,3 @@
[submodule "plugins/odds-ticker"]
path = plugins/odds-ticker
url = https://github.com/ChuckBuilds/ledmatrix-odds-ticker.git
[submodule "plugins/clock-simple"]
path = plugins/clock-simple
url = https://github.com/ChuckBuilds/ledmatrix-clock-simple.git
[submodule "plugins/text-display"]
path = plugins/text-display
url = https://github.com/ChuckBuilds/ledmatrix-text-display.git
[submodule "rpi-rgb-led-matrix-master"] [submodule "rpi-rgb-led-matrix-master"]
path = rpi-rgb-led-matrix-master path = rpi-rgb-led-matrix-master
url = https://github.com/hzeller/rpi-rgb-led-matrix.git url = https://github.com/hzeller/rpi-rgb-led-matrix.git
[submodule "plugins/basketball-scoreboard"]
path = plugins/basketball-scoreboard
url = https://github.com/ChuckBuilds/ledmatrix-basketball-scoreboard.git
[submodule "plugins/soccer-scoreboard"]
path = plugins/soccer-scoreboard
url = https://github.com/ChuckBuilds/ledmatrix-soccer-scoreboard.git
[submodule "plugins/calendar"]
path = plugins/calendar
url = https://github.com/ChuckBuilds/ledmatrix-calendar.git
[submodule "plugins/mqtt-notifications"]
path = plugins/mqtt-notifications
url = https://github.com/ChuckBuilds/ledmatrix-mqtt-notifications.git
[submodule "plugins/olympics-countdown"]
path = plugins/olympics-countdown
url = https://github.com/ChuckBuilds/ledmatrix-olympics-countdown.git
[submodule "plugins/ledmatrix-stocks"]
path = plugins/ledmatrix-stocks
url = https://github.com/ChuckBuilds/ledmatrix-stocks.git
[submodule "plugins/ledmatrix-music"]
path = plugins/ledmatrix-music
url = https://github.com/ChuckBuilds/ledmatrix-music.git
[submodule "plugins/static-image"]
path = plugins/static-image
url = https://github.com/ChuckBuilds/ledmatrix-static-image.git
[submodule "plugins/football-scoreboard"]
path = plugins/football-scoreboard
url = https://github.com/ChuckBuilds/ledmatrix-football-scoreboard.git
[submodule "plugins/hockey-scoreboard"]
path = plugins/hockey-scoreboard
url = https://github.com/ChuckBuilds/ledmatrix-hockey-scoreboard.git
[submodule "plugins/baseball-scoreboard"]
path = plugins/baseball-scoreboard
url = https://github.com/ChuckBuilds/ledmatrix-baseball-scoreboard.git
[submodule "plugins/christmas-countdown"]
path = plugins/christmas-countdown
url = https://github.com/ChuckBuilds/ledmatrix-christmas-countdown.git
[submodule "plugins/ledmatrix-flights"]
path = plugins/ledmatrix-flights
url = https://github.com/ChuckBuilds/ledmatrix-flights.git
[submodule "plugins/ledmatrix-leaderboard"]
path = plugins/ledmatrix-leaderboard
url = https://github.com/ChuckBuilds/ledmatrix-leaderboard.git
[submodule "plugins/ledmatrix-weather"]
path = plugins/ledmatrix-weather
url = https://github.com/ChuckBuilds/ledmatrix-weather.git
[submodule "plugins/ledmatrix-news"]
path = plugins/ledmatrix-news
url = https://github.com/ChuckBuilds/ledmatrix-news.git
[submodule "plugins/ledmatrix-of-the-day"]
path = plugins/ledmatrix-of-the-day
url = https://github.com/ChuckBuilds/ledmatrix-of-the-day.git
[submodule "plugins/youtube-stats"]
path = plugins/youtube-stats
url = https://github.com/ChuckBuilds/ledmatrix-youtube-stats.git

31
CLAUDE.md Normal file
View File

@@ -0,0 +1,31 @@
# LEDMatrix
## Project Structure
- `src/plugin_system/` — Plugin loader, manager, store manager, base plugin class
- `web_interface/` — Flask web UI (blueprints, templates, static JS)
- `config/config.json` — User plugin configuration (persists across plugin reinstalls)
- `plugins/` — Installed plugins directory (gitignored)
- `plugin-repos/` — Development symlinks to monorepo plugin dirs
## Plugin System
- Plugins inherit from `BasePlugin` in `src/plugin_system/base_plugin.py`
- Required abstract methods: `update()`, `display(force_clear=False)`
- Each plugin needs: `manifest.json`, `config_schema.json`, `manager.py`, `requirements.txt`
- Plugin instantiation args: `plugin_id, config, display_manager, cache_manager, plugin_manager`
- Config schemas use JSON Schema Draft-7
- Display dimensions: always read dynamically from `self.display_manager.matrix.width/height`
## Plugin Store Architecture
- Official plugins live in the `ledmatrix-plugins` monorepo (not individual repos)
- Plugin repo naming convention: `ledmatrix-<plugin-id>` (e.g., `ledmatrix-football-scoreboard`)
- `plugins.json` registry at `https://raw.githubusercontent.com/ChuckBuilds/ledmatrix-plugins/main/plugins.json`
- Store manager (`src/plugin_system/store_manager.py`) handles install/update/uninstall
- Monorepo plugins are installed via ZIP extraction (no `.git` directory)
- Update detection for monorepo plugins uses version comparison (manifest version vs registry latest_version)
- Plugin configs stored in `config/config.json`, NOT in plugin directories — safe across reinstalls
- Third-party plugins can use their own repo URL with empty `plugin_path`
## Common Pitfalls
- paho-mqtt 2.x needs `callback_api_version=mqtt.CallbackAPIVersion.VERSION1` for v1 compat
- BasePlugin uses `get_logger()` from `src.logging_config`, not standard `logging.getLogger()`
- When modifying a plugin in the monorepo, you MUST bump `version` in its `manifest.json` and run `python update_registry.py` — otherwise users won't receive the update

View File

@@ -4,89 +4,9 @@
"path": ".", "path": ".",
"name": "LEDMatrix (Main)" "name": "LEDMatrix (Main)"
}, },
{
"path": "../ledmatrix-odds-ticker",
"name": "Odds Ticker"
},
{
"path": "../ledmatrix-clock-simple",
"name": "Clock Simple"
},
{
"path": "../ledmatrix-text-display",
"name": "Text Display"
},
{
"path": "../ledmatrix-basketball-scoreboard",
"name": "Basketball Scoreboard"
},
{
"path": "../ledmatrix-soccer-scoreboard",
"name": "Soccer Scoreboard"
},
{
"path": "../ledmatrix-calendar",
"name": "Calendar"
},
{
"path": "../ledmatrix-olympics-countdown",
"name": "Olympics Countdown"
},
{
"path": "../ledmatrix-stocks",
"name": "Stocks"
},
{
"path": "../ledmatrix-music",
"name": "Music"
},
{
"path": "../ledmatrix-static-image",
"name": "Static Image"
},
{
"path": "../ledmatrix-football-scoreboard",
"name": "Football Scoreboard"
},
{
"path": "../ledmatrix-hockey-scoreboard",
"name": "Hockey Scoreboard"
},
{
"path": "../ledmatrix-baseball-scoreboard",
"name": "Baseball Scoreboard"
},
{
"path": "../ledmatrix-christmas-countdown",
"name": "Christmas Countdown"
},
{
"path": "../ledmatrix-flights",
"name": "Flights"
},
{
"path": "../ledmatrix-leaderboard",
"name": "Leaderboard"
},
{
"path": "../ledmatrix-weather",
"name": "Weather"
},
{
"path": "../ledmatrix-news",
"name": "News"
},
{
"path": "../ledmatrix-of-the-day",
"name": "Of The Day"
},
{
"path": "../ledmatrix-youtube-stats",
"name": "YouTube Stats"
},
{ {
"path": "../ledmatrix-plugins", "path": "../ledmatrix-plugins",
"name": "Plugin Registry" "name": "Plugins (Monorepo)"
} }
], ],
"settings": { "settings": {

View File

@@ -1,343 +0,0 @@
#!/bin/bash
# Script to normalize all plugins as git submodules
# This ensures uniform plugin management across the repository
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
PLUGINS_DIR="$PROJECT_ROOT/plugins"
GITMODULES="$PROJECT_ROOT/.gitmodules"
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
log_info() {
echo -e "${BLUE}[INFO]${NC} $1"
}
log_success() {
echo -e "${GREEN}[SUCCESS]${NC} $1"
}
log_warn() {
echo -e "${YELLOW}[WARN]${NC} $1"
}
log_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
# Check if a plugin is in .gitmodules
is_in_gitmodules() {
local plugin_path="$1"
git config -f "$GITMODULES" --get-regexp "^submodule\." | grep -q "path = $plugin_path$" || return 1
}
# Get submodule URL from .gitmodules
get_submodule_url() {
local plugin_path="$1"
git config -f "$GITMODULES" "submodule.$plugin_path.url" 2>/dev/null || echo ""
}
# Check if directory is a git repo
is_git_repo() {
[[ -d "$1/.git" ]]
}
# Get git remote URL
get_git_remote() {
local plugin_dir="$1"
if is_git_repo "$plugin_dir"; then
(cd "$plugin_dir" && git remote get-url origin 2>/dev/null || echo "")
else
echo ""
fi
}
# Check if directory is a symlink
is_symlink() {
[[ -L "$1" ]]
}
# Check if plugin has GitHub repo
has_github_repo() {
local plugin_name="$1"
local url="https://github.com/ChuckBuilds/ledmatrix-$plugin_name"
local status=$(curl -s -o /dev/null -w "%{http_code}" "$url" 2>/dev/null || echo "0")
[[ "$status" == "200" ]]
}
# Update .gitignore to allow a plugin submodule
update_gitignore() {
local plugin_name="$1"
local plugin_path="plugins/$plugin_name"
local gitignore="$PROJECT_ROOT/.gitignore"
# Check if already in .gitignore exceptions
if grep -q "!plugins/$plugin_name$" "$gitignore" 2>/dev/null; then
log_info "Plugin $plugin_name already in .gitignore exceptions"
return 0
fi
# Find the line with the last plugin exception
local last_line=$(grep -n "!plugins/" "$gitignore" | tail -1 | cut -d: -f1)
if [[ -z "$last_line" ]]; then
log_warn "Could not find plugin exceptions in .gitignore"
return 1
fi
# Add exceptions after the last plugin exception
log_info "Updating .gitignore to allow $plugin_name submodule"
sed -i "${last_line}a!plugins/$plugin_name\n!plugins/$plugin_name/" "$gitignore"
log_success "Updated .gitignore for $plugin_name"
}
# Re-initialize a submodule that appears as regular directory
reinit_submodule() {
local plugin_name="$1"
local plugin_path="plugins/$plugin_name"
local plugin_dir="$PLUGINS_DIR/$plugin_name"
log_info "Re-initializing submodule: $plugin_name"
if ! is_in_gitmodules "$plugin_path"; then
log_error "Plugin $plugin_name is not in .gitmodules"
return 1
fi
local submodule_url=$(get_submodule_url "$plugin_path")
if [[ -z "$submodule_url" ]]; then
log_error "Could not find URL for $plugin_name in .gitmodules"
return 1
fi
# If it's a symlink, remove it first
if is_symlink "$plugin_dir"; then
log_warn "Removing symlink: $plugin_dir"
rm "$plugin_dir"
fi
# If it's a regular directory with .git, we need to handle it carefully
if is_git_repo "$plugin_dir"; then
local remote_url=$(get_git_remote "$plugin_dir")
if [[ "$remote_url" == "$submodule_url" ]] || [[ "$remote_url" == "${submodule_url%.git}" ]] || [[ "${submodule_url%.git}" == "$remote_url" ]]; then
log_info "Directory is already the correct git repo, re-initializing submodule..."
# Remove from git index and re-add as submodule
git rm --cached "$plugin_path" 2>/dev/null || true
rm -rf "$plugin_dir"
else
log_warn "Directory has different remote ($remote_url vs $submodule_url)"
log_warn "Backing up to ${plugin_dir}.backup"
mv "$plugin_dir" "${plugin_dir}.backup"
fi
fi
# Re-add as submodule (use -f to force if needed)
if git submodule add -f "$submodule_url" "$plugin_path" 2>/dev/null; then
log_info "Submodule added successfully"
else
log_info "Submodule already exists, updating..."
git submodule update --init "$plugin_path"
fi
log_success "Re-initialized submodule: $plugin_name"
}
# Convert standalone git repo to submodule
convert_to_submodule() {
local plugin_name="$1"
local plugin_path="plugins/$plugin_name"
local plugin_dir="$PLUGINS_DIR/$plugin_name"
log_info "Converting to submodule: $plugin_name"
if is_in_gitmodules "$plugin_path"; then
log_warn "Plugin $plugin_name is already in .gitmodules, re-initializing instead"
reinit_submodule "$plugin_name"
return 0
fi
if ! is_git_repo "$plugin_dir"; then
log_error "Plugin $plugin_name is not a git repository"
return 1
fi
local remote_url=$(get_git_remote "$plugin_dir")
if [[ -z "$remote_url" ]]; then
log_error "Plugin $plugin_name has no remote URL"
return 1
fi
# If it's a symlink, we need to handle it differently
if is_symlink "$plugin_dir"; then
local target=$(readlink -f "$plugin_dir")
log_warn "Plugin is a symlink to $target"
log_warn "Removing symlink and adding as submodule"
rm "$plugin_dir"
# Update .gitignore first
update_gitignore "$plugin_name"
# Add as submodule
if git submodule add -f "$remote_url" "$plugin_path"; then
log_success "Added submodule: $plugin_name"
return 0
else
log_error "Failed to add submodule"
return 1
fi
fi
# Backup the directory
log_info "Backing up existing directory to ${plugin_dir}.backup"
mv "$plugin_dir" "${plugin_dir}.backup"
# Remove from git index
git rm --cached "$plugin_path" 2>/dev/null || true
# Update .gitignore first
update_gitignore "$plugin_name"
# Add as submodule (use -f to force if .gitignore blocks it)
if git submodule add -f "$remote_url" "$plugin_path"; then
log_success "Converted to submodule: $plugin_name"
log_warn "Backup saved at ${plugin_dir}.backup - you can remove it after verifying"
else
log_error "Failed to add submodule"
log_warn "Restoring backup..."
mv "${plugin_dir}.backup" "$plugin_dir"
return 1
fi
}
# Add new submodule for plugin with GitHub repo
add_new_submodule() {
local plugin_name="$1"
local plugin_path="plugins/$plugin_name"
local plugin_dir="$PLUGINS_DIR/$plugin_name"
local repo_url="https://github.com/ChuckBuilds/ledmatrix-$plugin_name.git"
log_info "Adding new submodule: $plugin_name"
if is_in_gitmodules "$plugin_path"; then
log_warn "Plugin $plugin_name is already in .gitmodules"
return 0
fi
if [[ -e "$plugin_dir" ]]; then
if is_symlink "$plugin_dir"; then
log_warn "Removing symlink: $plugin_dir"
rm "$plugin_dir"
elif is_git_repo "$plugin_dir"; then
log_warn "Directory exists as git repo, converting instead"
convert_to_submodule "$plugin_name"
return 0
else
log_warn "Backing up existing directory to ${plugin_dir}.backup"
mv "$plugin_dir" "${plugin_dir}.backup"
fi
fi
# Remove from git index if it exists
git rm --cached "$plugin_path" 2>/dev/null || true
# Update .gitignore first
update_gitignore "$plugin_name"
# Add as submodule (use -f to force if .gitignore blocks it)
if git submodule add -f "$repo_url" "$plugin_path"; then
log_success "Added new submodule: $plugin_name"
else
log_error "Failed to add submodule"
if [[ -e "${plugin_dir}.backup" ]]; then
log_warn "Restoring backup..."
mv "${plugin_dir}.backup" "$plugin_dir"
fi
return 1
fi
}
# Main processing function
main() {
cd "$PROJECT_ROOT"
log_info "Normalizing all plugins as git submodules..."
echo
# Step 1: Re-initialize submodules that appear as regular directories
log_info "Step 1: Re-initializing existing submodules..."
for plugin in basketball-scoreboard calendar clock-simple odds-ticker olympics-countdown soccer-scoreboard text-display mqtt-notifications; do
if [[ -d "$PLUGINS_DIR/$plugin" ]] && is_in_gitmodules "plugins/$plugin"; then
if ! git submodule status "plugins/$plugin" >/dev/null 2>&1; then
reinit_submodule "$plugin"
else
log_info "Submodule $plugin is already properly initialized"
fi
fi
done
echo
# Step 2: Convert standalone git repos to submodules
log_info "Step 2: Converting standalone git repos to submodules..."
for plugin in baseball-scoreboard ledmatrix-stocks; do
if [[ -d "$PLUGINS_DIR/$plugin" ]] && is_git_repo "$PLUGINS_DIR/$plugin"; then
if ! is_in_gitmodules "plugins/$plugin"; then
convert_to_submodule "$plugin"
fi
fi
done
echo
# Step 2b: Convert symlinks to submodules
log_info "Step 2b: Converting symlinks to submodules..."
for plugin in christmas-countdown ledmatrix-music static-image; do
if [[ -L "$PLUGINS_DIR/$plugin" ]]; then
if ! is_in_gitmodules "plugins/$plugin"; then
convert_to_submodule "$plugin"
fi
fi
done
echo
# Step 3: Add new submodules for plugins with GitHub repos
log_info "Step 3: Adding new submodules for plugins with GitHub repos..."
for plugin in football-scoreboard hockey-scoreboard; do
if [[ -d "$PLUGINS_DIR/$plugin" ]] && has_github_repo "$plugin"; then
if ! is_in_gitmodules "plugins/$plugin"; then
add_new_submodule "$plugin"
fi
fi
done
echo
# Step 4: Report on plugins without GitHub repos
log_info "Step 4: Checking plugins without GitHub repos..."
for plugin in ledmatrix-flights ledmatrix-leaderboard ledmatrix-weather; do
if [[ -d "$PLUGINS_DIR/$plugin" ]]; then
if ! is_in_gitmodules "plugins/$plugin" && ! is_git_repo "$PLUGINS_DIR/$plugin"; then
log_warn "Plugin $plugin has no GitHub repo and is not a git repo"
log_warn " This plugin may be local-only or needs a repository created"
fi
fi
done
echo
# Final: Initialize all submodules
log_info "Finalizing: Initializing all submodules..."
git submodule update --init --recursive
log_success "Plugin normalization complete!"
log_info "Run 'git status' to see changes"
log_info "Run 'git submodule status' to verify all submodules"
}
main "$@"

View File

@@ -1,151 +1,89 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Setup plugin repository references for multi-root workspace. Setup plugin repository symlinks for local development.
This script creates symlinks in plugin-repos/ pointing to the actual Creates symlinks in plugin-repos/ pointing to plugin directories
plugin repositories in the parent directory, allowing the system to in the ledmatrix-plugins monorepo.
find plugins without modifying the LEDMatrix project structure.
""" """
import json import json
import os import os
import re
import sys import sys
from pathlib import Path from pathlib import Path
# Paths
PROJECT_ROOT = Path(__file__).parent.parent PROJECT_ROOT = Path(__file__).parent.parent
PLUGIN_REPOS_DIR = PROJECT_ROOT / "plugin-repos" PLUGIN_REPOS_DIR = PROJECT_ROOT / "plugin-repos"
GITHUB_DIR = PROJECT_ROOT.parent MONOREPO_PLUGINS = PROJECT_ROOT.parent / "ledmatrix-plugins" / "plugins"
CONFIG_FILE = PROJECT_ROOT / "config" / "config.json"
def get_workspace_plugins(): def parse_json_with_trailing_commas(text: str) -> dict:
"""Get list of plugins from workspace file.""" """Parse JSON that may have trailing commas."""
workspace_file = PROJECT_ROOT / "LEDMatrix.code-workspace" text = re.sub(r",\s*([}\]])", r"\1", text)
if not workspace_file.exists(): return json.loads(text)
return []
try:
with open(workspace_file, 'r') as f:
workspace = json.load(f)
except json.JSONDecodeError as e:
print(f"Error: Failed to parse workspace file {workspace_file}: {e}")
print("Please check that the workspace file contains valid JSON.")
return []
plugins = []
for folder in workspace.get('folders', []):
path = folder.get('path', '')
if path.startswith('../') and path != '../ledmatrix-plugins':
plugin_name = path.replace('../', '')
plugins.append({
'name': plugin_name,
'workspace_path': path,
'actual_path': GITHUB_DIR / plugin_name,
'link_path': PLUGIN_REPOS_DIR / plugin_name
})
return plugins
def create_symlinks(): def create_symlinks() -> bool:
"""Create symlinks in plugin-repos/ pointing to actual repos.""" """Create symlinks in plugin-repos/ pointing to monorepo plugin dirs."""
plugins = get_workspace_plugins() if not MONOREPO_PLUGINS.exists():
print(f"Error: Monorepo plugins directory not found: {MONOREPO_PLUGINS}")
if not plugins:
print("No plugins found in workspace configuration")
return False return False
# Ensure plugin-repos directory exists
PLUGIN_REPOS_DIR.mkdir(exist_ok=True) PLUGIN_REPOS_DIR.mkdir(exist_ok=True)
created = 0 created = 0
skipped = 0 skipped = 0
errors = 0
print("Setting up plugin symlinks...")
print(f"Setting up plugin repository links...") print(f" Source: {MONOREPO_PLUGINS}")
print(f" Source: {GITHUB_DIR}")
print(f" Links: {PLUGIN_REPOS_DIR}") print(f" Links: {PLUGIN_REPOS_DIR}")
print() print()
for plugin in plugins: for plugin_dir in sorted(MONOREPO_PLUGINS.iterdir()):
actual_path = plugin['actual_path'] if not plugin_dir.is_dir():
link_path = plugin['link_path']
if not actual_path.exists():
print(f" ⚠️ {plugin['name']} - source not found: {actual_path}")
errors += 1
continue continue
manifest_path = plugin_dir / "manifest.json"
# Remove existing link/file if it exists if not manifest_path.exists():
continue
try:
with open(manifest_path, "r", encoding="utf-8") as f:
manifest = parse_json_with_trailing_commas(f.read())
except (OSError, json.JSONDecodeError) as e:
print(f" {plugin_dir.name} - failed to read {manifest_path}: {e}")
continue
plugin_id = manifest.get("id", plugin_dir.name)
link_path = PLUGIN_REPOS_DIR / plugin_id
if link_path.exists() or link_path.is_symlink(): if link_path.exists() or link_path.is_symlink():
if link_path.is_symlink(): if link_path.is_symlink():
# Check if it points to the right place
try: try:
if link_path.resolve() == actual_path.resolve(): if link_path.resolve() == plugin_dir.resolve():
print(f"{plugin['name']} - link already exists")
skipped += 1 skipped += 1
continue continue
else: else:
# Remove old symlink pointing elsewhere
link_path.unlink() link_path.unlink()
except Exception as e: except OSError:
print(f" ⚠️ {plugin['name']} - error checking link: {e}")
link_path.unlink() link_path.unlink()
else: else:
# It's a directory/file, not a symlink print(f" {plugin_id} - exists but is not a symlink, skipping")
print(f" ⚠️ {plugin['name']} - {link_path.name} exists but is not a symlink")
print(f" Skipping (manual cleanup required)")
skipped += 1 skipped += 1
continue continue
# Create symlink
try:
# Use relative path for symlink portability
relative_path = os.path.relpath(actual_path, link_path.parent)
link_path.symlink_to(relative_path)
print(f"{plugin['name']} - linked")
created += 1
except Exception as e:
print(f"{plugin['name']} - failed to create link: {e}")
errors += 1
print()
print(f"✅ Created {created} links, skipped {skipped}, errors {errors}")
return errors == 0
relative_path = os.path.relpath(plugin_dir, link_path.parent)
link_path.symlink_to(relative_path)
print(f" {plugin_id} - linked")
created += 1
def update_config_path(): print(f"\nCreated {created} links, skipped {skipped}")
"""Update config to use absolute path to parent directory (alternative approach).""" return True
# This is an alternative - set plugins_directory to absolute path
# Currently not implemented as symlinks are preferred
pass
def main(): def main():
"""Main function.""" print("Setting up plugin repository symlinks from monorepo...\n")
print("🔗 Setting up plugin repository symlinks...") if not create_symlinks():
print() sys.exit(1)
if not GITHUB_DIR.exists():
print(f"Error: GitHub directory not found: {GITHUB_DIR}")
return 1
success = create_symlinks()
if success:
print()
print("✅ Plugin repository setup complete!")
print()
print("Plugins are now accessible via symlinks in plugin-repos/")
print("You can update plugins independently in their git repos.")
return 0
else:
print()
print("⚠️ Setup completed with some errors. Check output above.")
return 1
if __name__ == '__main__': if __name__ == "__main__":
sys.exit(main()) main()

View File

@@ -1,123 +1,43 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Update all plugin repositories by pulling the latest changes. Update the ledmatrix-plugins monorepo by pulling latest changes.
This script updates all plugin repos without needing to modify
the LEDMatrix project itself.
""" """
import json
import os
import subprocess import subprocess
import sys import sys
from pathlib import Path from pathlib import Path
# Paths MONOREPO_DIR = Path(__file__).parent.parent.parent / "ledmatrix-plugins"
WORKSPACE_FILE = Path(__file__).parent.parent / "LEDMatrix.code-workspace"
GITHUB_DIR = Path(__file__).parent.parent.parent
def load_workspace_plugins():
"""Load plugin paths from workspace file."""
try:
with open(WORKSPACE_FILE, 'r', encoding='utf-8') as f:
workspace = json.load(f)
except FileNotFoundError:
print(f"Error: Workspace file not found: {WORKSPACE_FILE}")
return []
except PermissionError as e:
print(f"Error: Permission denied reading workspace file {WORKSPACE_FILE}: {e}")
return []
except json.JSONDecodeError as e:
print(f"Error: Invalid JSON in workspace file {WORKSPACE_FILE}: {e}")
return []
plugins = []
for folder in workspace.get('folders', []):
path = folder.get('path', '')
name = folder.get('name', '')
# Only process plugin folders (those starting with ../)
if path.startswith('../') and path != '../ledmatrix-plugins':
plugin_name = path.replace('../', '')
plugin_path = GITHUB_DIR / plugin_name
if plugin_path.exists():
plugins.append({
'name': plugin_name,
'display_name': name,
'path': plugin_path
})
return plugins
def update_repo(repo_path):
"""Update a git repository by pulling latest changes."""
if not (repo_path / '.git').exists():
print(f" ⚠️ {repo_path.name} is not a git repository, skipping")
return False
try:
# Fetch latest changes
fetch_result = subprocess.run(['git', 'fetch', 'origin'],
cwd=repo_path, capture_output=True, text=True)
if fetch_result.returncode != 0:
print(f" ✗ Failed to fetch {repo_path.name}: {fetch_result.stderr.strip()}")
return False
# Get current branch
branch_result = subprocess.run(['git', 'rev-parse', '--abbrev-ref', 'HEAD'],
cwd=repo_path, capture_output=True, text=True)
current_branch = branch_result.stdout.strip() if branch_result.returncode == 0 else 'main'
# Pull latest changes
pull_result = subprocess.run(['git', 'pull', 'origin', current_branch],
cwd=repo_path, capture_output=True, text=True)
if pull_result.returncode == 0:
# Check if there were actual updates
if 'Already up to date' in pull_result.stdout:
print(f"{repo_path.name} is up to date")
else:
print(f" ✓ Updated {repo_path.name}")
return True
else:
print(f" ✗ Failed to update {repo_path.name}: {pull_result.stderr.strip()}")
return False
except (subprocess.SubprocessError, OSError) as e:
print(f" ✗ Error updating {repo_path.name}: {e}")
return False
def main(): def main():
"""Main function.""" if not MONOREPO_DIR.exists():
print("🔍 Finding plugin repositories...") print(f"Error: Monorepo not found: {MONOREPO_DIR}")
plugins = load_workspace_plugins()
if not plugins:
print(" No plugin repositories found!")
return 1 return 1
print(f" Found {len(plugins)} plugin repositories") if not (MONOREPO_DIR / ".git").exists():
print(f"\n🚀 Updating plugins in {GITHUB_DIR}...") print(f"Error: {MONOREPO_DIR} is not a git repository")
print() return 1
success_count = 0 print(f"Updating {MONOREPO_DIR}...")
for plugin in plugins: try:
print(f"Updating {plugin['name']}...") result = subprocess.run(
if update_repo(plugin['path']): ["git", "-C", str(MONOREPO_DIR), "pull"],
success_count += 1 capture_output=True,
print() text=True,
timeout=120,
print(f"\n✅ Updated {success_count}/{len(plugins)} plugins successfully!") )
except subprocess.TimeoutExpired:
if success_count < len(plugins): print(f"Error: git pull timed out after 120 seconds for {MONOREPO_DIR}")
print("⚠️ Some plugins failed to update. Check the errors above.") return 1
if result.returncode == 0:
print(result.stdout.strip())
return 0
else:
print(f"Error: {result.stderr.strip()}")
return 1 return 1
return 0
if __name__ == '__main__': if __name__ == "__main__":
sys.exit(main()) sys.exit(main())

View File

@@ -560,7 +560,9 @@ class PluginStoreManager:
enhanced_plugin['last_commit_branch'] = commit_info.get('branch') enhanced_plugin['last_commit_branch'] = commit_info.get('branch')
# Fetch manifest from GitHub for additional metadata (description, etc.) # Fetch manifest from GitHub for additional metadata (description, etc.)
github_manifest = self._fetch_manifest_from_github(repo_url, branch) plugin_subpath = plugin.get('plugin_path', '')
manifest_rel = f"{plugin_subpath}/manifest.json" if plugin_subpath else "manifest.json"
github_manifest = self._fetch_manifest_from_github(repo_url, branch, manifest_rel)
if github_manifest: if github_manifest:
if 'last_updated' in github_manifest and not enhanced_plugin.get('last_updated'): if 'last_updated' in github_manifest and not enhanced_plugin.get('last_updated'):
enhanced_plugin['last_updated'] = github_manifest['last_updated'] enhanced_plugin['last_updated'] = github_manifest['last_updated']
@@ -571,14 +573,16 @@ class PluginStoreManager:
return results return results
def _fetch_manifest_from_github(self, repo_url: str, branch: str = "master") -> Optional[Dict]: def _fetch_manifest_from_github(self, repo_url: str, branch: str = "master", manifest_path: str = "manifest.json") -> Optional[Dict]:
""" """
Fetch manifest.json directly from a GitHub repository. Fetch manifest.json directly from a GitHub repository.
Args: Args:
repo_url: GitHub repository URL repo_url: GitHub repository URL
branch: Branch name (default: master) branch: Branch name (default: master)
manifest_path: Path to manifest within the repo (default: manifest.json).
For monorepo plugins this will be e.g. "plugins/football-scoreboard/manifest.json".
Returns: Returns:
Manifest data or None if not found Manifest data or None if not found
""" """
@@ -590,27 +594,27 @@ class PluginStoreManager:
repo_url = repo_url.rstrip('/') repo_url = repo_url.rstrip('/')
if repo_url.endswith('.git'): if repo_url.endswith('.git'):
repo_url = repo_url[:-4] repo_url = repo_url[:-4]
parts = repo_url.split('/') parts = repo_url.split('/')
if len(parts) >= 2: if len(parts) >= 2:
owner = parts[-2] owner = parts[-2]
repo = parts[-1] repo = parts[-1]
raw_url = f"https://raw.githubusercontent.com/{owner}/{repo}/{branch}/manifest.json" raw_url = f"https://raw.githubusercontent.com/{owner}/{repo}/{branch}/{manifest_path}"
response = self._http_get_with_retries(raw_url, timeout=10) response = self._http_get_with_retries(raw_url, timeout=10)
if response.status_code == 200: if response.status_code == 200:
return response.json() return response.json()
elif response.status_code == 404: elif response.status_code == 404:
# Try main branch instead # Try main branch instead
if branch != "main": if branch != "main":
raw_url = f"https://raw.githubusercontent.com/{owner}/{repo}/main/manifest.json" raw_url = f"https://raw.githubusercontent.com/{owner}/{repo}/main/{manifest_path}"
response = self._http_get_with_retries(raw_url, timeout=10) response = self._http_get_with_retries(raw_url, timeout=10)
if response.status_code == 200: if response.status_code == 200:
return response.json() return response.json()
except Exception as e: except Exception as e:
self.logger.debug(f"Could not fetch manifest from GitHub for {repo_url}: {e}") self.logger.debug(f"Could not fetch manifest from GitHub for {repo_url}: {e}")
return None return None
def _get_latest_commit_info(self, repo_url: str, branch: str = "main") -> Optional[Dict[str, Any]]: def _get_latest_commit_info(self, repo_url: str, branch: str = "main") -> Optional[Dict[str, Any]]:
@@ -722,7 +726,9 @@ class PluginStoreManager:
plugin_info['branch'] = commit_info.get('branch', branch) plugin_info['branch'] = commit_info.get('branch', branch)
plugin_info['last_commit_branch'] = commit_info.get('branch') plugin_info['last_commit_branch'] = commit_info.get('branch')
github_manifest = self._fetch_manifest_from_github(repo_url, branch) plugin_subpath = plugin_info.get('plugin_path', '')
manifest_rel = f"{plugin_subpath}/manifest.json" if plugin_subpath else "manifest.json"
github_manifest = self._fetch_manifest_from_github(repo_url, branch, manifest_rel)
if github_manifest: if github_manifest:
if 'last_updated' in github_manifest and not plugin_info.get('last_updated'): if 'last_updated' in github_manifest and not plugin_info.get('last_updated'):
plugin_info['last_updated'] = github_manifest['last_updated'] plugin_info['last_updated'] = github_manifest['last_updated']
@@ -1128,76 +1134,247 @@ class PluginStoreManager:
def _install_from_monorepo(self, download_url: str, plugin_subpath: str, target_path: Path) -> bool: def _install_from_monorepo(self, download_url: str, plugin_subpath: str, target_path: Path) -> bool:
""" """
Install a plugin from a monorepo by downloading and extracting a subdirectory. Install a plugin from a monorepo by downloading only the target subdirectory.
Uses the GitHub Git Trees API to list files, then downloads each file
individually from raw.githubusercontent.com. Falls back to downloading
the full ZIP archive if the API approach fails.
Args: Args:
download_url: URL to download zip from download_url: URL to download zip from (used as fallback and to extract repo info)
plugin_subpath: Path within repo (e.g., "plugins/hello-world") plugin_subpath: Path within repo (e.g., "plugins/hello-world")
target_path: Target directory for plugin target_path: Target directory for plugin
Returns: Returns:
True if successful True if successful
""" """
# Try the API-based approach first (downloads only the target directory)
repo_url, branch = self._parse_monorepo_download_url(download_url)
if repo_url and branch:
result = self._install_from_monorepo_api(repo_url, branch, plugin_subpath, target_path)
if result:
return True
self.logger.info(f"API-based install failed for {plugin_subpath}, falling back to ZIP download")
# Ensure no partial files remain before ZIP fallback
if target_path.exists():
shutil.rmtree(target_path, ignore_errors=True)
# Fallback: download full ZIP and extract subdirectory
return self._install_from_monorepo_zip(download_url, plugin_subpath, target_path)
@staticmethod
def _parse_monorepo_download_url(download_url: str):
"""Extract repo URL and branch from a GitHub archive download URL.
Example: "https://github.com/ChuckBuilds/ledmatrix-plugins/archive/refs/heads/main.zip"
Returns: ("https://github.com/ChuckBuilds/ledmatrix-plugins", "main")
"""
try: try:
self.logger.info(f"Downloading monorepo from: {download_url}") # Pattern: {repo_url}/archive/refs/heads/{branch}.zip
if '/archive/refs/heads/' in download_url:
parts = download_url.split('/archive/refs/heads/')
repo_url = parts[0]
branch = parts[1].removesuffix('.zip')
return repo_url, branch
except (IndexError, AttributeError):
pass
return None, None
@staticmethod
def _normalize_repo_url(url: str) -> str:
"""Normalize a GitHub repo URL for comparison (strip trailing / and .git)."""
url = url.rstrip('/')
if url.endswith('.git'):
url = url[:-4]
return url.lower()
def _install_from_monorepo_api(self, repo_url: str, branch: str, plugin_subpath: str, target_path: Path) -> bool:
"""
Install a plugin subdirectory using the GitHub Git Trees API.
Downloads only the files in the target subdirectory (~200KB) instead
of the entire repository ZIP (~5MB+). Uses one API call for the tree
listing, then downloads individual files from raw.githubusercontent.com.
Args:
repo_url: GitHub repository URL (e.g., "https://github.com/owner/repo")
branch: Branch name (e.g., "main")
plugin_subpath: Path within repo (e.g., "plugins/hello-world")
target_path: Target directory for plugin
Returns:
True if successful, False to trigger ZIP fallback
"""
try:
# Parse owner/repo from URL
clean_url = repo_url.rstrip('/')
if clean_url.endswith('.git'):
clean_url = clean_url[:-4]
parts = clean_url.split('/')
if len(parts) < 2:
return False
owner, repo = parts[-2], parts[-1]
# Step 1: Get the recursive tree listing (1 API call)
api_url = f"https://api.github.com/repos/{owner}/{repo}/git/trees/{branch}?recursive=true"
headers = {
'Accept': 'application/vnd.github.v3+json',
'User-Agent': 'LEDMatrix-Plugin-Manager/1.0'
}
if self.github_token:
headers['Authorization'] = f'token {self.github_token}'
tree_response = self._http_get_with_retries(api_url, timeout=15, headers=headers)
if tree_response.status_code != 200:
self.logger.debug(f"Trees API returned {tree_response.status_code} for {owner}/{repo}")
return False
tree_data = tree_response.json()
if tree_data.get('truncated'):
self.logger.debug(f"Tree response truncated for {owner}/{repo}, falling back to ZIP")
return False
# Step 2: Filter for files in the target subdirectory
prefix = f"{plugin_subpath}/"
file_entries = [
entry for entry in tree_data.get('tree', [])
if entry['path'].startswith(prefix) and entry['type'] == 'blob'
]
if not file_entries:
self.logger.error(f"No files found under '{plugin_subpath}' in tree for {owner}/{repo}")
return False
# Sanity check: refuse unreasonably large plugin directories
max_files = 500
if len(file_entries) > max_files:
self.logger.error(
f"Plugin {plugin_subpath} has {len(file_entries)} files (limit {max_files}), "
f"falling back to ZIP"
)
return False
self.logger.info(f"Downloading {len(file_entries)} files for {plugin_subpath} via API")
# Step 3: Create target directory and download each file
from src.common.permission_utils import (
ensure_directory_permissions,
get_plugin_dir_mode
)
ensure_directory_permissions(target_path.parent, get_plugin_dir_mode())
target_path.mkdir(parents=True, exist_ok=True)
prefix_len = len(prefix)
target_root = target_path.resolve()
for entry in file_entries:
# Relative path within the plugin directory
rel_path = entry['path'][prefix_len:]
dest_file = target_path / rel_path
# Guard against path traversal
if not dest_file.resolve().is_relative_to(target_root):
self.logger.error(
f"Path traversal detected: {entry['path']!r} resolves outside target directory"
)
if target_path.exists():
shutil.rmtree(target_path, ignore_errors=True)
return False
# Create parent directories
dest_file.parent.mkdir(parents=True, exist_ok=True)
# Download from raw.githubusercontent.com (no API rate limit cost)
raw_url = f"https://raw.githubusercontent.com/{owner}/{repo}/{branch}/{entry['path']}"
file_response = self._http_get_with_retries(raw_url, timeout=30)
if file_response.status_code != 200:
self.logger.error(f"Failed to download {entry['path']}: HTTP {file_response.status_code}")
# Clean up partial download
if target_path.exists():
shutil.rmtree(target_path, ignore_errors=True)
return False
dest_file.write_bytes(file_response.content)
self.logger.info(f"Successfully installed {plugin_subpath} via API ({len(file_entries)} files)")
return True
except Exception as e:
self.logger.debug(f"API-based monorepo install failed: {e}")
# Clean up partial download
if target_path.exists():
shutil.rmtree(target_path, ignore_errors=True)
return False
def _install_from_monorepo_zip(self, download_url: str, plugin_subpath: str, target_path: Path) -> bool:
"""
Fallback: install a plugin from a monorepo by downloading the full ZIP.
Used when the API-based approach fails (rate limited, auth issues, etc.).
"""
tmp_zip_path = None
temp_extract = None
try:
self.logger.info(f"Downloading monorepo ZIP from: {download_url}")
response = self._http_get_with_retries(download_url, timeout=60, stream=True) response = self._http_get_with_retries(download_url, timeout=60, stream=True)
response.raise_for_status() response.raise_for_status()
# Download to temporary file # Download to temporary file
with tempfile.NamedTemporaryFile(suffix='.zip', delete=False) as tmp_file: with tempfile.NamedTemporaryFile(suffix='.zip', delete=False) as tmp_file:
for chunk in response.iter_content(chunk_size=8192): for chunk in response.iter_content(chunk_size=8192):
tmp_file.write(chunk) tmp_file.write(chunk)
tmp_zip_path = tmp_file.name tmp_zip_path = tmp_file.name
try: with zipfile.ZipFile(tmp_zip_path, 'r') as zip_ref:
# Extract zip zip_contents = zip_ref.namelist()
with zipfile.ZipFile(tmp_zip_path, 'r') as zip_ref: if not zip_contents:
zip_contents = zip_ref.namelist() return False
if not zip_contents:
return False root_dir = zip_contents[0].split('/')[0]
plugin_prefix = f"{root_dir}/{plugin_subpath}/"
# GitHub zips have a root directory like "repo-main/"
root_dir = zip_contents[0].split('/')[0] # Extract ONLY files under the plugin subdirectory
plugin_members = [m for m in zip_contents if m.startswith(plugin_prefix)]
# Build path to plugin within extracted archive
# e.g., "ledmatrix-plugins-main/plugins/hello-world/" if not plugin_members:
plugin_path_in_zip = f"{root_dir}/{plugin_subpath}/" self.logger.error(f"Plugin path not found in archive: {plugin_subpath}")
return False
# Extract to temp location
temp_extract = Path(tempfile.mkdtemp()) temp_extract = Path(tempfile.mkdtemp())
zip_ref.extractall(temp_extract) temp_extract_resolved = temp_extract.resolve()
# Find the plugin directory for member in plugin_members:
source_plugin_dir = temp_extract / root_dir / plugin_subpath # Guard against zip-slip (directory traversal)
member_dest = (temp_extract / member).resolve()
if not source_plugin_dir.exists(): if not member_dest.is_relative_to(temp_extract_resolved):
self.logger.error(f"Plugin path not found in archive: {plugin_subpath}") self.logger.error(
self.logger.error(f"Expected at: {source_plugin_dir}") f"Zip-slip detected: member {member!r} resolves outside "
shutil.rmtree(temp_extract, ignore_errors=True) f"temp directory, skipping"
return False )
continue
# Move plugin contents to target zip_ref.extract(member, temp_extract)
from src.common.permission_utils import (
ensure_directory_permissions, source_plugin_dir = temp_extract / root_dir / plugin_subpath
get_plugin_dir_mode
) from src.common.permission_utils import (
ensure_directory_permissions(target_path.parent, get_plugin_dir_mode()) ensure_directory_permissions,
shutil.move(str(source_plugin_dir), str(target_path)) get_plugin_dir_mode
)
# Cleanup temp extract dir ensure_directory_permissions(target_path.parent, get_plugin_dir_mode())
if temp_extract.exists(): # Ensure target doesn't exist to prevent shutil.move nesting
shutil.rmtree(temp_extract, ignore_errors=True) if target_path.exists():
shutil.rmtree(target_path, ignore_errors=True)
return True shutil.move(str(source_plugin_dir), str(target_path))
finally: return True
# Remove temporary zip file
if os.path.exists(tmp_zip_path):
os.remove(tmp_zip_path)
except Exception as e: except Exception as e:
self.logger.error(f"Monorepo download failed: {e}", exc_info=True) self.logger.error(f"Monorepo ZIP download failed: {e}", exc_info=True)
return False return False
finally:
if tmp_zip_path and os.path.exists(tmp_zip_path):
os.remove(tmp_zip_path)
if temp_extract and temp_extract.exists():
shutil.rmtree(temp_extract, ignore_errors=True)
def _install_via_download(self, download_url: str, target_path: Path) -> bool: def _install_via_download(self, download_url: str, target_path: Path) -> bool:
""" """
@@ -1535,22 +1712,37 @@ class PluginStoreManager:
# Plugin is a git repository - try to update via git # Plugin is a git repository - try to update via git
local_branch = git_info.get('branch') or 'main' local_branch = git_info.get('branch') or 'main'
local_sha = git_info.get('sha') local_sha = git_info.get('sha')
# Try to get remote info from registry (optional) # Try to get remote info from registry (optional)
self.fetch_registry(force_refresh=True) self.fetch_registry(force_refresh=True)
plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True) plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True)
remote_branch = None remote_branch = None
remote_sha = None remote_sha = None
if plugin_info_remote: if plugin_info_remote:
remote_branch = plugin_info_remote.get('branch') or plugin_info_remote.get('default_branch') remote_branch = plugin_info_remote.get('branch') or plugin_info_remote.get('default_branch')
remote_sha = plugin_info_remote.get('last_commit_sha') remote_sha = plugin_info_remote.get('last_commit_sha')
# Check if the local git remote still matches the registry repo URL.
# After monorepo migration, old clones point to archived individual repos
# while the registry now points to the monorepo. Detect this and reinstall.
registry_repo = plugin_info_remote.get('repo', '')
local_remote = git_info.get('remote_url', '')
if local_remote and registry_repo and self._normalize_repo_url(local_remote) != self._normalize_repo_url(registry_repo):
self.logger.info(
f"Plugin {plugin_id} git remote ({local_remote}) differs from registry ({registry_repo}). "
f"Reinstalling from registry to migrate to new source."
)
if not self._safe_remove_directory(plugin_path):
self.logger.error(f"Failed to remove old plugin directory for {plugin_id}")
return False
return self.install_plugin(plugin_id)
# Check if already up to date # Check if already up to date
if remote_sha and local_sha and remote_sha.startswith(local_sha): if remote_sha and local_sha and remote_sha.startswith(local_sha):
self.logger.info(f"Plugin {plugin_id} already matches remote commit {remote_sha[:7]}") self.logger.info(f"Plugin {plugin_id} already matches remote commit {remote_sha[:7]}")
return True return True
# Update via git pull # Update via git pull
self.logger.info(f"Updating {plugin_id} via git pull (local branch: {local_branch})...") self.logger.info(f"Updating {plugin_id} via git pull (local branch: {local_branch})...")
try: try:
@@ -1833,7 +2025,22 @@ class PluginStoreManager:
remote_sha = plugin_info_remote.get('last_commit_sha') remote_sha = plugin_info_remote.get('last_commit_sha')
remote_branch = plugin_info_remote.get('branch') or plugin_info_remote.get('default_branch') remote_branch = plugin_info_remote.get('branch') or plugin_info_remote.get('default_branch')
# If we get here, plugin is not a git repo but is in registry - reinstall # Compare local manifest version against registry latest_version
# to avoid unnecessary reinstalls for monorepo plugins
try:
local_manifest_path = plugin_path / "manifest.json"
if local_manifest_path.exists():
with open(local_manifest_path, 'r', encoding='utf-8') as f:
local_manifest = json.load(f)
local_version = local_manifest.get('version', '')
remote_version = plugin_info_remote.get('latest_version', '')
if local_version and remote_version and local_version == remote_version:
self.logger.info(f"Plugin {plugin_id} already at latest version {local_version}")
return True
except Exception as e:
self.logger.debug(f"Could not compare versions for {plugin_id}: {e}")
# Plugin is not a git repo but is in registry and has a newer version - reinstall
self.logger.info(f"Plugin {plugin_id} not installed via git; re-installing latest archive") self.logger.info(f"Plugin {plugin_id} not installed via git; re-installing latest archive")
# Remove directory and reinstall fresh # Remove directory and reinstall fresh

View File

@@ -1793,6 +1793,7 @@ def get_installed_plugins():
plugins.append({ plugins.append({
'id': plugin_id, 'id': plugin_id,
'name': plugin_info.get('name', plugin_id), 'name': plugin_info.get('name', plugin_id),
'version': plugin_info.get('version', ''),
'author': plugin_info.get('author', 'Unknown'), 'author': plugin_info.get('author', 'Unknown'),
'category': plugin_info.get('category', 'General'), 'category': plugin_info.get('category', 'General'),
'description': plugin_info.get('description', 'No description available'), 'description': plugin_info.get('description', 'No description available'),
@@ -3168,8 +3169,10 @@ def list_plugin_store():
'last_commit': plugin.get('last_commit') or plugin.get('last_commit_sha'), 'last_commit': plugin.get('last_commit') or plugin.get('last_commit_sha'),
'last_commit_message': plugin.get('last_commit_message'), 'last_commit_message': plugin.get('last_commit_message'),
'last_commit_author': plugin.get('last_commit_author'), 'last_commit_author': plugin.get('last_commit_author'),
'version': plugin.get('latest_version') or plugin.get('version', ''),
'branch': plugin.get('branch') or plugin.get('default_branch'), 'branch': plugin.get('branch') or plugin.get('default_branch'),
'default_branch': plugin.get('default_branch') 'default_branch': plugin.get('default_branch'),
'plugin_path': plugin.get('plugin_path', '')
}) })
return jsonify({'status': 'success', 'data': {'plugins': formatted_plugins}}) return jsonify({'status': 'success', 'data': {'plugins': formatted_plugins}})

View File

@@ -1403,10 +1403,8 @@ function renderInstalledPlugins(plugins) {
</div> </div>
<div class="text-sm text-gray-600 space-y-1.5 mb-3"> <div class="text-sm text-gray-600 space-y-1.5 mb-3">
<p class="flex items-center"><i class="fas fa-user mr-2 text-gray-400 w-4"></i>${escapeHtml(plugin.author || 'Unknown')}</p> <p class="flex items-center"><i class="fas fa-user mr-2 text-gray-400 w-4"></i>${escapeHtml(plugin.author || 'Unknown')}</p>
<p class="flex items-center"><i class="fas fa-code-branch mr-2 text-gray-400 w-4"></i>${formatCommit(plugin.last_commit, plugin.branch)}</p> ${plugin.version ? `<p class="flex items-center"><i class="fas fa-tag mr-2 text-gray-400 w-4"></i>v${escapeHtml(plugin.version)}</p>` : ''}
<p class="flex items-center"><i class="fas fa-calendar mr-2 text-gray-400 w-4"></i>${formatDate(plugin.last_updated)}</p>
<p class="flex items-center"><i class="fas fa-folder mr-2 text-gray-400 w-4"></i>${escapeHtml(plugin.category || 'General')}</p> <p class="flex items-center"><i class="fas fa-folder mr-2 text-gray-400 w-4"></i>${escapeHtml(plugin.category || 'General')}</p>
${plugin.stars ? `<p class="flex items-center"><i class="fas fa-star mr-2 text-gray-400 w-4"></i>${plugin.stars} stars</p>` : ''}
</div> </div>
<p class="text-sm text-gray-700 leading-relaxed">${escapeHtml(plugin.description || 'No description available')}</p> <p class="text-sm text-gray-700 leading-relaxed">${escapeHtml(plugin.description || 'No description available')}</p>
</div> </div>
@@ -5263,10 +5261,8 @@ function renderPluginStore(plugins) {
</div> </div>
<div class="text-sm text-gray-600 space-y-1.5 mb-3"> <div class="text-sm text-gray-600 space-y-1.5 mb-3">
<p class="flex items-center"><i class="fas fa-user mr-2 text-gray-400 w-4"></i>${escapeHtml(plugin.author || 'Unknown')}</p> <p class="flex items-center"><i class="fas fa-user mr-2 text-gray-400 w-4"></i>${escapeHtml(plugin.author || 'Unknown')}</p>
<p class="flex items-center"><i class="fas fa-code-branch mr-2 text-gray-400 w-4"></i>${formatCommit(plugin.last_commit, plugin.branch)}</p> ${plugin.version ? `<p class="flex items-center"><i class="fas fa-tag mr-2 text-gray-400 w-4"></i>v${escapeHtml(plugin.version)}</p>` : ''}
<p class="flex items-center"><i class="fas fa-calendar mr-2 text-gray-400 w-4"></i>${formatDate(plugin.last_updated)}</p> <p class="flex items-center"><i class="fas fa-folder mr-2 text-gray-400 w-4"></i>${escapeHtml(plugin.category || 'General')}</p>
<p class="flex items-center"><i class="fas fa-tag mr-2 text-gray-400 w-4"></i>${escapeHtml(plugin.category || 'General')}</p>
<p class="flex items-center"><i class="fas fa-star mr-2 text-gray-400 w-4"></i>${plugin.stars || 0} stars</p>
</div> </div>
<p class="text-sm text-gray-700 leading-relaxed">${escapeHtml(plugin.description || 'No description available')}</p> <p class="text-sm text-gray-700 leading-relaxed">${escapeHtml(plugin.description || 'No description available')}</p>
</div> </div>
@@ -5293,7 +5289,7 @@ function renderPluginStore(plugins) {
<button onclick='if(window.installPlugin){const branchInput = document.getElementById("branch-input-${plugin.id.replace(/[^a-zA-Z0-9]/g, '-')}"); window.installPlugin(${escapeJs(plugin.id)}, branchInput?.value?.trim() || null)}else{console.error("installPlugin not available")}' class="btn bg-green-600 hover:bg-green-700 text-white px-4 py-2 rounded-md text-sm flex-1 font-semibold"> <button onclick='if(window.installPlugin){const branchInput = document.getElementById("branch-input-${plugin.id.replace(/[^a-zA-Z0-9]/g, '-')}"); window.installPlugin(${escapeJs(plugin.id)}, branchInput?.value?.trim() || null)}else{console.error("installPlugin not available")}' class="btn bg-green-600 hover:bg-green-700 text-white px-4 py-2 rounded-md text-sm flex-1 font-semibold">
<i class="fas fa-download mr-2"></i>Install <i class="fas fa-download mr-2"></i>Install
</button> </button>
<button onclick='window.open(${escapeJs(plugin.repo || '#')}, "_blank")' class="btn bg-gray-600 hover:bg-gray-700 text-white px-4 py-2 rounded-md text-sm flex-1 font-semibold"> <button onclick='${plugin.repo ? `window.open(${escapeJs(plugin.plugin_path ? plugin.repo + "/tree/" + (plugin.default_branch || plugin.branch || "main") + "/" + encodeURI(plugin.plugin_path) : plugin.repo)}, "_blank")` : `void(0)`}' ${plugin.repo ? '' : 'disabled'} class="btn bg-gray-600 hover:bg-gray-700 text-white px-4 py-2 rounded-md text-sm flex-1 font-semibold${plugin.repo ? '' : ' opacity-50 cursor-not-allowed'}">
<i class="fas fa-external-link-alt mr-2"></i>View <i class="fas fa-external-link-alt mr-2"></i>View
</button> </button>
</div> </div>