rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-13 10:36:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(deps): describe how plugin dependencies are really installed

Browse Source 
The guides said the web service runs as root, that installs pick --user
from os.geteuid(), and quoted a warning and a
PluginManager._install_plugin_dependencies() method that don't exist. The
web unit runs as the installing user; store installs go through
install_requirements_file() and sudo safe_pip_install.sh (root), with a
user-level fallback that says so, and load-time installs run in the
display service's own (root) interpreter.

Manual paths now use the configured plugins directory (plugin-repos/ by
default) instead of plugins/, which store installs no longer use, and
install_plugin_dependencies.sh is described as scanning that directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-17 08:58:03 -04:00
co-authored by Claude Opus 5
parent df3524305f
commit c3465c35f5
2 changed files with 187 additions and 266 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/PLUGIN_DEPENDENCY_GUIDE.md
+109 -183
View File
@@ -2,234 +2,160 @@
## Overview ## Overview
The LEDMatrix system has smart dependency installation that adapts based on who is running it. This guide explains how it works and potential pitfalls. A plugin lists its Python packages in its `requirements.txt`. LEDMatrix
installs them for you when a plugin is installed, updated or loaded. This
guide explains where they end up and what to do when a plugin can't import a
package.
## How It Works The rule to remember: **packages must be importable by `ledmatrix.service`,
which runs as root.** Anything installed only into another user's
`~/.local/` is invisible to it.
### Execution Context Detection ## Who Runs What
The plugin manager checks if it's running as root: | Service | Runs as | Set by |
```python |---------|---------|--------|
running_as_root = os.geteuid() == 0 | `ledmatrix.service` (display) | `root` | `systemd/ledmatrix.service` |
| `ledmatrix-web.service` (web UI) | the user who ran the installer (e.g. `ledpi`) | `User=__USER__` in `systemd/ledmatrix-web.service`, filled in by `scripts/install/install_service.sh` |
## How Dependencies Get Installed
### 1. Installing or updating a plugin from the web UI
The web interface is not root, so it installs through a narrow sudo helper:
1. `PluginStoreManager._install_dependencies()`
(`src/plugin_system/store_manager.py`) calls
`install_requirements_file()` (`src/common/permission_utils.py`).
2. That runs `sudo -n bash scripts/fix_perms/safe_pip_install.sh <plugin>/requirements.txt`.
The helper checks the path is the project's own `requirements.txt` or a
`requirements.txt` under `plugin-repos/` or `plugins/`, then runs
`python3 -m pip install --break-system-packages --ignore-installed -r ...`
**as root**, so the display service can import the packages.
3. The sudoers rule that allows this is written by the installer
(`first_time_install.sh`) or by `scripts/install/configure_web_sudo.sh`.
If sudo refuses (the rule isn't installed), `install_requirements_file()`
falls back to installing with the web process's own interpreter, as the web
user, and prefixes the pip output with a note like:
```
[Root install unavailable (...); installed for the current process's user only.
Packages may not be visible to ledmatrix.service if it runs as a different
user — run scripts/install/configure_web_sudo.sh to fix this.]
``` ```
Based on this, it chooses the appropriate installation method: Fix it by running `./scripts/install/configure_web_sudo.sh` as the web
user (not with `sudo`; it asks for your password itself), then
reinstall the plugin (or use the manual install below).
| Running As | Installation Method | Location | Accessible To | The **Reinstall Plugin Deps** button on the web UI's Tools tab goes
|------------|-------------------|----------|---------------| through the same helper for every installed plugin.
| **root** (systemd service) | System-wide (`--break-system-packages`) | `/usr/local/lib/python3.X/dist-packages/` | All users |
| **ledpi** or other user | User-specific (`--user`) | `~/.local/lib/python3.X/site-packages/` | Only that user | ### 2. Loading a plugin
When a plugin loads, `PluginLoader.install_dependencies()`
(`src/plugin_system/plugin_loader.py`) checks its `requirements.txt`. If the
requirements are already satisfied it does nothing; otherwise it runs
`python3 -m pip install --break-system-packages -r requirements.txt` with the
interpreter of the process doing the loading (retrying with
`--ignore-installed` when a system package without a pip RECORD file is in
the way).
In `ledmatrix.service` that process is root, so restarting the display
service installs anything missing system-wide:
```bash
sudo systemctl restart ledmatrix
```
If you run `python3 run.py` by hand as a normal user instead, pip cannot
write to the system site-packages and installs into your `~/.local/`. That
works for your manual run but not for the service.
## Common Scenarios ## Common Scenarios
### ✅ Scenario 1: Normal Production Use (Recommended) ### Installing plugins from the web UI (recommended)
**What:** Services running via systemd Use the **Plugin Manager** tab. Dependencies are installed as root through
the sudo helper and the display service can use them.
### Running the display manually for debugging
```bash ```bash
sudo systemctl start ledmatrix cd ~/LEDMatrix
sudo systemctl start ledmatrix-web sudo python3 run.py # same user as the service
``` ```
- **Runs as:** root (configured in .service files) Running as your own user works for plugins whose packages are already
- **Installs to:** System-wide installed system-wide, but any *missing* package lands in `~/.local/`.
- **Result:** ✅ Works perfectly, all dependencies accessible
### ✅ Scenario 2: Web Interface Plugin Installation ### A plugin works when run manually but fails in the service
**What:** Installing/enabling plugins via web interface at `http://pi-ip:5000` Its packages were installed for your user only. Install them as root (see
below) and restart the service.
- **Web service runs as:** root (ledmatrix-web.service) ## Manual Installation
- **Installs to:** System-wide
- **Result:** ✅ Works perfectly, systemd service can access them
### ✅ Scenario 3: Manual Testing as ledpi (Read-only) ### All plugins
**What:** Running display manually as ledpi to test/debug
```bash ```bash
# As ledpi user sudo ~/LEDMatrix/scripts/install_plugin_dependencies.sh
cd /home/ledpi/LEDMatrix
python3 run.py
```
- **Runs as:** ledpi
- **Can import:** ✅ System-wide packages (installed by root)
- **Result:** ✅ Works! Can use existing plugins with root-installed dependencies
### ⚠️ Scenario 4: Manual Plugin Installation as ledpi (Problematic)
**What:** Enabling a NEW plugin and running manually as ledpi
```bash
# As ledpi user
cd /home/ledpi/LEDMatrix
# Edit config to enable new plugin
nano config/config.json
# Run display - will try to install new plugin dependencies
python3 run.py
```
**What Happens:**
1. Plugin manager runs as `ledpi`
2. Installs dependencies with `--user` flag
3. Dependencies go to `~/.local/lib/python3.X/site-packages/`
4. ⚠️ **Warning logged:** "Installing plugin dependencies for current user (not root)"
**Problem:**
- When systemd service restarts (as root), it **can't see** `~/.local/` packages
- Plugin will fail to load for the systemd service
**Solution:**
After testing, restart the service to install dependencies system-wide:
```bash
sudo systemctl restart ledmatrix sudo systemctl restart ledmatrix
``` ```
## Best Practices The script installs every `requirements.txt` found in the plugins directory
configured by `plugin_system.plugins_directory` in `config/config.json`
(default `plugin-repos/`). Run it with `sudo` so the packages are installed
system-wide.
### For Production/Normal Use ### One plugin
1. **Always use the web interface** to install/enable plugins
2. **Or restart the systemd service** after config changes:
```bash ```bash
cd ~/LEDMatrix/plugin-repos/PLUGIN-NAME # or your configured plugins directory
sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
sudo systemctl restart ledmatrix sudo systemctl restart ledmatrix
``` ```
### For Development/Testing `--no-cache-dir` avoids errors about `/root/.cache/pip` not being writable.
1. **Read existing plugins:** Safe to run as `ledpi` - can import system packages
2. **Test new plugins:** Use sudo or restart service to install dependencies:
```bash
# Option 1: Run as root
sudo python3 run.py
# Option 2: Install deps manually
sudo pip3 install --break-system-packages -r plugins/my-plugin/requirements.txt
python3 run.py
# Option 3: Let service install them
sudo systemctl restart ledmatrix
```
## Warning Messages
### If you see this warning:
```
Installing plugin dependencies for current user (not root).
These will NOT be accessible to the systemd service.
For production use, install plugins via the web interface or restart the ledmatrix service.
```
**What it means:**
- You're running as a non-root user
- Dependencies were installed to your user directory only
- The systemd service won't be able to use this plugin
**What to do:**
```bash
# Restart the service to install dependencies system-wide
sudo systemctl restart ledmatrix
```
## Troubleshooting ## Troubleshooting
### Plugin works when I run manually but fails in systemd service
**Cause:** Dependencies installed to user directory (`~/.local/`) instead of system-wide
**Fix:**
```bash
# Check where package is installed
pip3 list -v | grep <package-name>
# If it shows ~/.local/, reinstall system-wide:
sudo pip3 install --break-system-packages <package-name>
# Or just restart the service:
sudo systemctl restart ledmatrix
```
### Permission denied when installing dependencies ### Permission denied when installing dependencies
**If you see errors like:**
``` ```
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/root/.local' ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/root/.local'
WARNING: The directory '/root/.cache/pip' or its parent directory is not owned or is not writable WARNING: The directory '/root/.cache/pip' or its parent directory is not owned or is not writable
``` ```
**Quick Fix - Use the Helper Script:** Use one of the manual installs above (they pass `--no-cache-dir`).
```bash
sudo /home/ledpi/LEDMatrix/scripts/install_plugin_dependencies.sh
sudo systemctl restart ledmatrix
```
**Manual Fix:** ### Checking where a package is installed
```bash
# Install dependencies with --no-cache-dir to avoid cache permission issues
cd /home/ledpi/LEDMatrix/plugins/PLUGIN-NAME
sudo pip3 install --break-system-packages --no-cache-dir -r requirements.txt
sudo systemctl restart ledmatrix
```
**For more detailed troubleshooting, see:** [Plugin Dependency Troubleshooting Guide](PLUGIN_DEPENDENCY_TROUBLESHOOTING.md)
## Architecture Summary
```
┌─────────────────────────────────────────────────────────────┐
│ LEDMatrix Services │
├─────────────────────────────────────────────────────────────┤
│ │
│ ledmatrix.service (User=root) │
│ ledmatrix-web.service (User=root) │
│ ├── Install dependencies system-wide │
│ └── Accessible to all users │
│ │
├─────────────────────────────────────────────────────────────┤
│ │
│ Manual execution as ledpi │
│ ├── Can READ system-wide packages ✅ │
│ ├── WRITES go to ~/.local/ ⚠️ │
│ └── Not accessible to root service │
│ │
└─────────────────────────────────────────────────────────────┘
```
## Recommendations
1. **For end users:** Always use the web interface for plugin management
2. **For developers:** Be aware of the user context when testing
3. **For plugin authors:** Test with `sudo systemctl restart ledmatrix` to ensure dependencies install correctly
4. **For CI/CD:** Always run installation as root or use the service
## Helper Scripts
### Install Plugin Dependencies Script
Located at: `scripts/install_plugin_dependencies.sh`
This script automatically finds and installs dependencies for all plugins:
```bash ```bash
# Run as root (recommended for production) # How the service sees it
sudo /home/ledpi/LEDMatrix/scripts/install_plugin_dependencies.sh sudo python3 -c "import package_name; print(package_name.__file__)"
# Make executable if needed # A path under /home/<user>/.local/ means it was installed for that user only
chmod +x /home/ledpi/LEDMatrix/scripts/install_plugin_dependencies.sh python3 -m pip show -f package_name
``` ```
Features: For more, see the [Plugin Dependency Troubleshooting Guide](PLUGIN_DEPENDENCY_TROUBLESHOOTING.md).
- Auto-detects all plugins with requirements.txt
- Uses correct installation method (system-wide vs user) ## For Plugin Authors
- Bypasses pip cache to avoid permission issues
- Provides detailed logging and error messages 1. Keep `requirements.txt` minimal and pin only what you need.
2. Test that it installs the way the Pi will install it:
```bash
sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
```
3. Note any `apt` packages your plugin needs in its README.
## Files to Reference ## Files to Reference
- Service configs: `ledmatrix.service`, `ledmatrix-web.service` - Service units: `systemd/ledmatrix.service`, `systemd/ledmatrix-web.service`
- Plugin manager: `src/plugin_system/plugin_manager.py` - Store installs: `src/plugin_system/store_manager.py` (`_install_dependencies`)
- Installation script: `first_time_install.sh` - Root install helper: `src/common/permission_utils.py` (`install_requirements_file`), `scripts/fix_perms/safe_pip_install.sh`
- Dependency installer: `scripts/install_plugin_dependencies.sh` - Load-time installs: `src/plugin_system/plugin_loader.py` (`install_dependencies`)
- Troubleshooting guide: `PLUGIN_DEPENDENCY_TROUBLESHOOTING.md` - Sudo rules: `scripts/install/configure_web_sudo.sh`
- Manual installer: `scripts/install_plugin_dependencies.sh`
docs/PLUGIN_DEPENDENCY_TROUBLESHOOTING.md
+77 -82
View File
@@ -1,6 +1,7 @@
# Plugin Dependency Installation Troubleshooting # Plugin Dependency Installation Troubleshooting
This guide helps resolve issues with automatic plugin dependency installation in the LEDMatrix system. This guide helps resolve problems installing a plugin's Python packages. For
how installation works, see the [Plugin Dependency Guide](PLUGIN_DEPENDENCY_GUIDE.md).
## Common Error Symptoms ## Common Error Symptoms
@@ -10,109 +11,118 @@ ERROR: Could not install packages due to an OSError: [Errno 13] Permission denie
WARNING: The directory '/root/.cache/pip' or its parent directory is not owned or is not writable WARNING: The directory '/root/.cache/pip' or its parent directory is not owned or is not writable
``` ```
### Context Mismatch ### Installed for the wrong user
The pip output shown after a web-UI install starts with:
``` ```
WARNING: Installing plugin dependencies for current user (not root). [Root install unavailable (...); installed for the current process's user only.
These will NOT be accessible to the systemd service. Packages may not be visible to ledmatrix.service if it runs as a different
user — run scripts/install/configure_web_sudo.sh to fix this.]
``` ```
### Plugin fails to load with `ModuleNotFoundError`
The display service can't see a package the plugin needs.
## Root Cause ## Root Cause
Plugin dependencies must be installed in a context accessible to the LEDMatrix systemd service, which runs as root. Permission errors typically occur when: Plugin packages must be importable by `ledmatrix.service`, which runs as
root. The web interface (`ledmatrix-web.service`) runs as the user who
installed LEDMatrix, so it installs through a sudo helper
(`scripts/fix_perms/safe_pip_install.sh`). Problems usually come from:
1. The pip cache directory has incorrect permissions 1. The sudoers rule for that helper missing, so the web UI installed the
2. The process tries to install to user directories without proper permissions packages for its own user only
3. Environment variables (like HOME) are not set correctly for the service context 2. Running `python3 run.py` by hand as a normal user, which installs missing
packages into `~/.local/`
3. pip's cache directory not being writable for root
## Solutions ## Solutions
### Solution 1: Use the Manual Installation Script (Recommended) ### Solution 1: Restore the sudo rule, then reinstall
We provide a helper script that handles dependency installation correctly:
```bash ```bash
# Run as root to install system-wide (for production) cd ~/LEDMatrix
sudo /home/ledpi/LEDMatrix/scripts/install_plugin_dependencies.sh ./scripts/install/configure_web_sudo.sh # as the web user, not with sudo
```
# After installation, restart the service Then reinstall the plugin from the **Plugin Manager** tab, or click
**Reinstall Plugin Deps** on the **Tools** tab.
### Solution 2: Install every plugin's dependencies from the terminal
```bash
sudo ~/LEDMatrix/scripts/install_plugin_dependencies.sh
sudo systemctl restart ledmatrix sudo systemctl restart ledmatrix
``` ```
This script: The script finds each `requirements.txt` in the plugins directory set by
- Detects all plugins with requirements.txt files `plugin_system.plugins_directory` in `config/config.json` (default
- Installs dependencies with correct permissions `plugin-repos/`), installs with `--no-cache-dir`, and reports what it found.
- Uses `--no-cache-dir` to avoid cache permission issues
- Provides detailed logging for troubleshooting
### Solution 2: Manual Installation per Plugin ### Solution 3: Install one plugin's dependencies
If you need to install dependencies for a specific plugin:
```bash ```bash
# Navigate to the plugin directory # Your configured plugins directory; plugin-repos/ by default
cd /home/ledpi/LEDMatrix/plugins/PLUGIN-NAME cd ~/LEDMatrix/plugin-repos/PLUGIN-NAME
# Install as root (system-wide) sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
sudo pip3 install --break-system-packages --no-cache-dir -r requirements.txt
# Restart the service
sudo systemctl restart ledmatrix sudo systemctl restart ledmatrix
``` ```
### Solution 3: Fix Cache Directory Permissions ### Solution 4: Let the display service install them
If you specifically have cache permission issues: When a plugin loads, the display service installs any missing requirements
itself, as root:
```bash
sudo systemctl restart ledmatrix
sudo journalctl -u ledmatrix -f # watch for "Installing dependencies for plugin ..."
```
### Solution 5: Fix pip cache permissions
```bash ```bash
# Option A: Skip the cache (recommended) # Option A: Skip the cache (recommended)
sudo pip3 install --no-cache-dir --break-system-packages -r requirements.txt sudo python3 -m pip install --no-cache-dir --break-system-packages -r requirements.txt
# Option B: Fix cache permissions (if needed) # Option B: Fix cache permissions
sudo mkdir -p /root/.cache/pip sudo mkdir -p /root/.cache/pip
sudo chown -R root:root /root/.cache sudo chown -R root:root /root/.cache
sudo chmod -R 755 /root/.cache sudo chmod -R 755 /root/.cache
``` ```
### Solution 4: Install via Web Interface
The web interface handles dependency installation correctly in the service context:
1. Access the web interface (`http://ledpi:5000` or `http://your-pi-ip:5000`)
2. Open the **Plugin Manager** tab (use the **Plugin Store** section to
find the plugin, or **Install from GitHub**)
3. Install the plugin through the web UI
4. The system automatically handles dependency installation in the
service context (which has the right permissions)
## Prevention ## Prevention
### For Plugin Developers ### For Plugin Developers
When creating plugins with dependencies:
1. **Keep requirements minimal**: Only include essential packages 1. **Keep requirements minimal**: Only include essential packages
2. **Test installation**: Verify your requirements.txt works with: 2. **Test installation** the way the Pi does it:
```bash ```bash
sudo pip3 install --break-system-packages --no-cache-dir -r requirements.txt sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
``` ```
3. **Document dependencies**: Note any system packages needed (via apt) 3. **Document dependencies**: Note any system packages needed (via apt)
### For Users ### For Users
1. **Use web interface**: Install plugins via the web UI when possible 1. **Use the web interface** to install plugins
2. **Install as root**: When using SSH/terminal, use sudo for plugin installations 2. **Use sudo** for installs from SSH/terminal
3. **Restart service**: After manual installations, restart the ledmatrix service 3. **Restart the service** after manual installations
## Technical Details ## Technical Details
### How Dependency Installation Works ### Where installs happen
The `PluginManager._install_plugin_dependencies()` method: - **Web UI install/update:** `PluginStoreManager._install_dependencies()`
→ `install_requirements_file()` in `src/common/permission_utils.py`, which
1. Detects if running as root using `os.geteuid() == 0` runs `sudo -n bash scripts/fix_perms/safe_pip_install.sh <requirements.txt>`.
2. If root: Uses system-wide installation with `--break-system-packages --no-cache-dir` The helper only accepts the project's `requirements.txt` or one under
3. If not root: Uses user installation with `--user --break-system-packages --no-cache-dir` `plugin-repos/` or `plugins/`, and runs
4. The `--no-cache-dir` flag prevents cache-related permission issues `pip install --break-system-packages --ignore-installed` as root. If sudo
refuses, it falls back to a pip install as the web user and says so.
- **Plugin load:** `PluginLoader.install_dependencies()` in
`src/plugin_system/plugin_loader.py` skips satisfied requirements and
otherwise runs `pip install --break-system-packages` with the loading
process's interpreter — root in `ledmatrix.service`.
### Why `--break-system-packages`? ### Why `--break-system-packages`?
@@ -120,26 +130,20 @@ Debian 12+ (Bookworm) and Raspberry Pi OS based on it implement PEP 668, which p
### Service Context ### Service Context
The ledmatrix.service runs as: - `ledmatrix.service` runs as **root** with `/usr/bin/python3`
- **User**: root - `ledmatrix-web.service` runs as **the installing user**
- **WorkingDirectory**: /home/ledpi/LEDMatrix
- **Python**: /usr/bin/python3
Dependencies must be installed in root's Python environment or system-wide to be accessible. Dependencies must be installed system-wide (as root) to be visible to the
display service.
## Checking Installation ## Checking Installation
Verify dependencies are installed correctly:
```bash ```bash
# Check as root (how the service sees it) # Check as root (how the service sees it)
sudo python3 -c "import package_name" sudo python3 -c "import package_name; print(package_name.__file__)"
# List installed packages # A path under /home/<user>/.local/ means a user-only install
pip3 list python3 -m pip show -f package_name
# Check specific package
pip3 show package_name
``` ```
## Getting Help ## Getting Help
@@ -151,19 +155,11 @@ If you continue to experience issues:
sudo journalctl -u ledmatrix -f sudo journalctl -u ledmatrix -f
``` ```
2. Check pip logs (created by manual script): 2. Verify the plugin manifest and requirements (default plugins directory
shown):
```bash ```bash
cat /tmp/pip_install_*.log cat ~/LEDMatrix/plugin-repos/PLUGIN-NAME/manifest.json
``` cat ~/LEDMatrix/plugin-repos/PLUGIN-NAME/requirements.txt
3. Verify plugin manifest is correct:
```bash
cat /home/ledpi/LEDMatrix/plugins/PLUGIN-NAME/manifest.json
```
4. Check plugin requirements:
```bash
cat /home/ledpi/LEDMatrix/plugins/PLUGIN-NAME/requirements.txt
``` ```
## Related Documentation ## Related Documentation
@@ -171,4 +167,3 @@ If you continue to experience issues:
- [Plugin Dependency Guide](PLUGIN_DEPENDENCY_GUIDE.md) - [Plugin Dependency Guide](PLUGIN_DEPENDENCY_GUIDE.md)
- [Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md) - [Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md)
- [Troubleshooting](TROUBLESHOOTING.md) - [Troubleshooting](TROUBLESHOOTING.md)