mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-08-01 08:48:05 +00:00
05e7c43b27
* fix(plugins): replace dependency marker files with a real satisfaction check The .dependencies_installed hash-marker system only tracked "was this exact requirements.txt hashed before" — not whether the packages it names are actually present. That made it fragile (a wiped venv, a manually removed package, or a lost/corrupted marker forces a needless full pip reinstall or, worse, a false skip) and produced dead weight for the ~10 plugins whose requirements.txt is comment-only (they still paid a pip subprocess on first boot before a marker existed). Replace it with requirements_are_satisfied() in plugin_loader.py, which checks each real requirement line against importlib.metadata directly, so install_dependencies() only shells out to pip when something is actually missing or version-mismatched. Drops the marker file entirely: removed all marker read/write sites in plugin_loader.py and store_manager.py, the now-pointless marker-cleanup step in the git-update path, the unused legacy marker implementation in plugin_manager.py, and the already-stale clear_dependency_markers.sh script. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(security): close path-injection gap in dependency-satisfaction checks CodeQL flagged 2 new high-severity "uncontrolled data used in path expression" alerts at the open() calls inside this PR's new requirements_has_real_deps()/requirements_are_satisfied() -- both are reachable from paths that were never run through the basename+trusted-base sanitiser this codebase already uses elsewhere: - PluginLoader.install_dependencies() only applied that sanitiser when its optional plugins_dir argument was actually passed; the "no plugins_dir" branch trusted plugin_dir_real directly. Made plugins_dir required (not Optional) so that branch can't exist, and added an explicit guard in load_plugin() so install_deps=True without a plugins_dir fails loudly instead of silently. Production's only real caller (PluginManager) always passes plugins_dir already; the harness/dev-server/render-plugin callers all use install_deps=False and are unaffected. - StoreManager._install_dependencies() never sanitised plugin_path at all, and its call sites ultimately derive that path from a plugin's own manifest.json "id" field (install_plugin_from_url) -- a malicious plugin could otherwise point requirements_file outside plugins_dir. Applied the same os.path.basename()-based containment pattern PluginLoader already uses (and that CodeQL recognises as a real sanitiser). Added test_install_dependencies_requires_plugins_dir and test_install_dependencies_rejects_path_outside_plugins_dir to lock in the actual security property, not just quiet the scanner. Verified: all 20 tests in test_plugin_loader.py pass, plus the PR's existing test plan (test_plugin_system.py, test_store_manager_caches.py: 53 passed) and the full CI plugin-safety suite (test_harness.py, test_visual_rendering.py, test_plugin_matrix.py: 52 passed, 2 pre-existing skips) all still pass. * fix(security): replace basename-only sanitiser with a trusted-enumeration check The previous commit's os.path.basename() + os.path.join() pattern (which a pre-existing code comment claimed CodeQL recognises as a sanitiser) did not actually clear the alert -- the next CodeQL run still flagged the same 2 sink lines, plus a new one at the os.path.join() call itself. Taking a substring of tainted data apparently isn't treated as a barrier by this query, whatever the comment assumed. Replaced it with find_trusted_subdir(): enumerate the trusted plugins_dir via os.scandir() and only use a name that scandir itself produced, matched by equality against the caller's requested name. The path is then built from that enumerated entry, not from the caller's string -- a value sourced from iterating a trusted, non-tainted directory carries no taint regardless of what it happens to equal, which is a stronger and more conventional allowlist-style barrier than string-stripping. Applied identically in both PluginLoader.install_dependencies() and StoreManager._install_dependencies(), sharing one implementation. Re-verified: all 65 tests across test_plugin_loader.py (20, including the 2 new security regression tests), test_store_manager_caches.py (35), test_plugin_system.py (10) pass, plus the full CI plugin-safety suite (test_harness.py/test_visual_rendering.py/test_plugin_matrix.py: 52 passed, 2 pre-existing skips). * fix(security): redact URL credentials from pip subprocess output before logging CodeQL flagged 3 clear-text-logging-of-secrets alerts in install_requirements_file() (src/common/permission_utils.py:353,360,371). Pre-existing on main, unrelated to this PR's own diff, but now visible since the path-injection alerts that previously took priority in the annotation list are fixed. The underlying risk is real: pip can echo a private index URL's embedded basic-auth credentials (from a requirements.txt --index-url line or PIP_INDEX_URL) back verbatim in its own stderr/stdout on failure, and this function both logs that output directly and returns it to callers -- store_manager.py's _install_dependencies() logs result.stderr from this same function too. Added _redact_url_credentials(), applied immediately after each of the two subprocess.run() calls (mutating result.stderr/stdout in place) rather than patching each log call site individually. This closes the leak at the source: every downstream use -- the three flagged log lines, the "note" string embedded in the returned stdout, and store_manager.py's own logging of the returned result -- gets the redacted text for free. Verified the fixed-phrase "denied" check (`"a password is required" in result.stderr`) is unaffected, since URL syntax and those phrases don't overlap -- covered explicitly by test_does_not_touch_denied_check_phrases. Added test/test_permission_utils.py (6 tests) covering the redaction helper directly and both subprocess.run() call sites (the sudo-wrapper branch, which this repo's scripts/fix_perms/safe_pip_install.sh makes live, and the no-wrapper fallback branch). All pass. * fix(security): stop interpolating req_file/pip-output into log calls The previous commit's redaction (mutating result.stderr/stdout right after each subprocess.run()) didn't clear CodeQL's clear-text-logging alerts -- same lesson as the path-injection fix earlier in this PR: a static analyzer can't tell "this value was already sanitised two lines up" from "this is still the raw tainted value" just by looking at a single log call in isolation, so it conservatively keeps flagging it regardless of what the redaction function actually does. Removed all dynamic interpolation (req_file, result.stderr) from the 3 flagged logger.warning() calls entirely, replacing them with fixed messages plus (for the one that had it) result.returncode, which is a plain int with no possible taint. The full redacted detail is still available where it actually matters -- in the returned CompletedProcess.stderr/stdout and the "note" text -- just not duplicated into a log line a scanner has to reason about in isolation. Re-verified: all 6 test_permission_utils.py tests still pass (they assert on the returned result, not log call arguments), plus the full test_plugin_loader.py/test_store_manager_caches.py/test_plugin_system.py suite (71 passed, 1 pre-existing deselect, 4 subtests). --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
76 lines
2.6 KiB
Plaintext
76 lines
2.6 KiB
Plaintext
# LEDMatrix Core Dependencies
|
|
# Compatible with Python 3.10, 3.11, 3.12, and 3.13
|
|
# Tested on Raspbian OS 12 (Bookworm) and 13 (Trixie)
|
|
|
|
# Image processing
|
|
Pillow>=12.2.0,<13.0.0
|
|
numpy>=1.24.0 # For fast array operations in ScrollHelper (compatible with 2.x)
|
|
|
|
# Timezone handling
|
|
pytz>=2024.2,<2025.0 # Updated for latest timezone data
|
|
timezonefinder>=6.5.0,<7.0.0 # Updated for better performance and accuracy
|
|
geopy>=2.4.1,<3.0.0
|
|
|
|
# HTTP requests
|
|
requests>=2.33.0,<3.0.0
|
|
|
|
# Google API integration
|
|
google-auth-oauthlib>=1.2.0,<2.0.0
|
|
google-auth-httplib2>=0.2.0,<1.0.0
|
|
google-api-python-client>=2.147.0,<3.0.0
|
|
|
|
# Font rendering
|
|
freetype-py>=2.5.1,<3.0.0
|
|
|
|
# Spotify integration
|
|
spotipy>=2.25.2,<3.0.0
|
|
|
|
# Flask web framework
|
|
Flask>=3.1.3,<4.0.0
|
|
|
|
# Text processing
|
|
unidecode>=1.3.8,<2.0.0
|
|
|
|
# Calendar integration
|
|
icalevents>=0.1.27,<1.0.0
|
|
|
|
# WebSocket support
|
|
python-socketio>=5.14.0,<6.0.0
|
|
python-engineio>=4.9.0,<5.0.0
|
|
websockets>=12.0,<14.0
|
|
websocket-client>=1.8.0,<2.0.0
|
|
|
|
# JSON Schema validation
|
|
jsonschema>=4.20.0,<5.0.0
|
|
|
|
# Requirement specifier parsing (plugin dependency satisfaction checks)
|
|
packaging>=23.0,<27.0
|
|
|
|
# Testing dependencies
|
|
pytest>=9.0.3,<10.0.0
|
|
pytest-cov>=4.1.0,<5.0.0
|
|
pytest-mock>=3.11.0,<4.0.0
|
|
mypy>=1.5.0,<2.0.0
|
|
|
|
# ───────────────────────────────────────────────────────────────────────
|
|
# Optional dependencies — the code imports these inside try/except
|
|
# blocks and gracefully degrades when missing. Install them for the
|
|
# full feature set, or skip them for a minimal install.
|
|
# ───────────────────────────────────────────────────────────────────────
|
|
#
|
|
# scipy — sub-pixel interpolation in
|
|
# src/common/scroll_helper.py for smoother
|
|
# scrolling. Falls back to a simpler shift algorithm.
|
|
# pip install 'scipy>=1.10.0,<2.0.0'
|
|
#
|
|
# psutil — per-plugin resource monitoring in
|
|
# src/plugin_system/resource_monitor.py. The monitor
|
|
# silently no-ops when missing (PSUTIL_AVAILABLE = False).
|
|
# pip install 'psutil>=5.9.0,<6.0.0'
|
|
#
|
|
# Flask-Limiter — request rate limiting in web_interface/app.py
|
|
# (accidental-abuse protection, not security). The
|
|
# web interface starts without rate limiting when
|
|
# this is missing.
|
|
# pip install 'Flask-Limiter>=3.5.0,<4.0.0'
|