refactor(web): delete dead routes, JS files and duplicate definitions (#609)

* refactor(web): drop validators nothing calls

escape_html, validate_image_url, validate_font_awesome_class,
validate_mime_type, validate_numeric_range, validate_string_length and
sanitize_plugin_config had no callers outside their own tests. Only
validate_file_upload (fonts upload) is imported by the web interface.

dedup_unique_arrays is kept: its one caller in save_plugin_config was
removed by the unrelated sync PR (#330), which looks accidental.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(api): remove the music-auth and of-the-day JSON routes

POST /plugins/authenticate/spotify and /plugins/authenticate/ytm had no
caller but their tests: the music plugin authenticates through its
web_ui_actions (authenticate_spotify.py / authenticate_ytm.py) via
/plugins/action.

POST /plugins/of-the-day/json/upload and /json/delete looked the plugin
up by the id ledmatrix-of-the-day (its manifest id is of-the-day), were
reachable only from a file_type "json" upload field that no schema
declares, and put the plugin directory on sys.path per request to
import scripts.update_config. of-the-day manages its files through
plugin-file-manager and its own web_ui_actions.

The of-the-day branch of GET /plugins/config stays: it matches the real
manifest id and still merges the on-disk category files into the form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(api): read managers only from the blueprints

api_v3/__init__.py and pages_v3.py declared module globals
(plugin_store_manager, saved_repositories_manager, schema_manager,
operation_queue, plugin_state_manager, operation_history, sync_manager,
config_manager, plugin_manager) that nothing assigns: app.py sets the
managers as attributes on the Blueprint objects, and every route reads
them there. The one reader, backup restore's fallback to the module
plugin_store_manager, could only ever fall back to None.

_ensure_cache_manager() built a second CacheManager in the web process
instead of using the one app.py puts on api_v3. The display routes now
read api_v3.cache_manager, creating it on the blueprint only when
nothing set it (the same None handling as the /cache routes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(web): drop run.sh and the unused log_config_change

web_interface/run.sh was referenced only by web_interface/README.md;
the service starts the UI through scripts/utils/start_web_conditionally.py
and the README already documents `python3 web_interface/start.py`.
log_config_change() in web_interface/logging_config.py was never called.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): delete unreferenced store_manager.js, diff_viewer.js, htmx-sse.js

- js/plugins/store_manager.js (window.PluginStoreManager) and
  js/config/diff_viewer.js (window.ConfigDiffViewer) were loaded on every
  page but nothing reads either global.
- htmx-sse.js (plus its CDN fallback) was loaded after HTMX, but no
  template or plugin page uses sse-connect / hx-ext="sse": the live
  streams run through LEDStreams in app-shell.js.

js/plugins/state_manager.js stays: install_manager.js's updateAll()
reads and refreshes window.PluginStateManager.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): remove app.js helpers nothing calls

- hexToRgb, rgbToHex, validateForm, uploadFont and switchTab (whose
  'switch-tab' event had no listener) have no caller in the templates,
  static JS or the plugin monorepo.
- installPlugin: plugins_manager.js (loaded last) assigns
  window.installPlugin, and its own store cards are the only callers.
- The showNotification fallback could never install: app-shell.js is
  deferred ahead of app.js and defines the same fallback at top level.
- performanceMonitor only logged with ?debug=perf and read an unset
  this.measures; the marks it took on every load had no reader.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): drop app-shell.js refreshPlugin

A top-level function in app-shell.js, so a window global, but nothing
calls it (no inline handler, no window lookup, no string-built name).

The other plugin actions in that block stay. updatePlugin is the live
window.updatePlugin: plugins_manager.js only installs its own copy when
none exists. uninstallPlugin/pollUninstallOperation, updateAllPlugins,
executePluginAction and toggleNestedSection are replaced by later
deferred scripts, but a click that lands while those scripts are still
downloading reaches the app-shell copies, so removing them is not a
pure no-op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): remove definitions plugins_manager.js always overrides

All of these are replaced before anything can call them, checked
against the load order in base.html and the live window.* values:

- openOnDemandModal/requestOnDemandStop stubs: the IIFE later in the
  same script assigns the real functions synchronously.
- updatePlugin and uninstallPlugin stubs (`window.X || stub`): app-shell.js
  already defined both, so the fallback never installed. Same for the
  later updatePlugin override, gated on the live function containing
  '[UPDATE]', which app-shell.js's never does.
- The first addArrayObjectItem/removeArrayObjectItem: reassigned by the
  top-level copies after the IIFE.
- The first `function formatDate` in the IIFE: a later declaration of
  the same name in the same scope wins.
- deleteUploadedImage, getCurrentImages, showUploadProgress,
  formatFileSize and getScheduleSummary: character-for-character
  copies of js/widgets/file-upload.js, which stays the owner.
- `typeof X === 'undefined'` fallbacks and `typeof X !== 'undefined'`
  re-exports after the IIFE: always false, or a self-assignment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): render the shell directly and delete index.html

index.html extended base.html with {% block content %}, but base.html
defines no blocks, so none of index.html ever rendered: rendering both
with jinja2 gives byte-identical output. index() still loaded the config,
read config.json and config_secrets.json raw and json.dumps'd them on
every page load for variables base.html never reads, and flashed errors
that base.html never shows. It now renders base.html with no context.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): stop htmx-config.js replacing console.error and console.warn

It swapped both globals for filters that dropped any error mentioning
insertBefore / "Cannot read properties of null" when "htmx" appeared in
the message or stack, and a list of Permissions-Policy warnings. That
hid real errors from every script on the page, and made every logged
error and warning report htmx-config.js as its source. The beforeSwap
target validation above it, which prevents the insertBefore errors in
the first place, stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(web): quiet the widget load announcements and debug logs

About 30 lines hit the console on every page load: one "... widget
registered" per widget file, one "[WidgetRegistry] Registered widget: X"
per registration, plus the registry, base widget and plugin loader
announcing themselves. The load-time announcements are removed; the
per-call ones (registry register, plugin widget loads, "Render called")
now go through the page's debugLog switch (localStorage.pluginDebug),
guarded because the widgets also load in node tests without it.
fonts.html and wifi.html debug logging goes through debugLog as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(api): drop the removed music-auth and of-the-day JSON routes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-23 12:36:53 -04:00
committed by GitHub
co-authored by Claude Opus 5.5
parent a231d4dbc7
commit a8b3e86775
62 changed files with 73 additions and 3149 deletions
+1 -394
View File
@@ -16,7 +16,7 @@ from web_interface.blueprints.api_v3 import (
api_v3, datetime, deep_merge, describe_exception, error_response,
find_secret_fields, hashlib, json, jsonify, logger, logging,
merge_secrets, os, redact_text, remove_empty_secrets, request,
separate_secrets, shutil, stat, subprocess, success_response, sys,
separate_secrets, shutil, stat, subprocess, success_response,
tempfile, uuid, validate_request_json,
)
from src.common.path_safety import (
@@ -3181,195 +3181,6 @@ sys.exit(proc.returncode)
except Exception as e:
logger.error('Error in execute_plugin_action', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/authenticate/spotify', methods=['POST'])
def authenticate_spotify():
"""Run Spotify authentication script"""
try:
data = request.get_json(silent=True) or {}
redirect_url = data.get('redirect_url', '').strip()
# Get plugin directory
plugin_id = 'ledmatrix-music'
if api_v3.plugin_manager:
plugin_dir = api_v3.plugin_manager.get_plugin_directory(plugin_id)
else:
plugin_dir = PROJECT_ROOT / 'plugins' / plugin_id
if not plugin_dir or not Path(plugin_dir).exists():
return jsonify({'status': 'error', 'message': 'Plugin not found'}), 404
auth_script = Path(plugin_dir) / 'authenticate_spotify.py'
if not auth_script.exists():
return jsonify({'status': 'error', 'message': 'Authentication script not found'}), 404
# Set LEDMATRIX_ROOT environment variable
env = os.environ.copy()
env['LEDMATRIX_ROOT'] = str(PROJECT_ROOT)
if redirect_url:
# Step 2: Complete authentication with redirect URL
# Create a wrapper script that provides the redirect URL as input
import tempfile
# Create a wrapper script that provides the redirect URL
import json
redirect_url_escaped = json.dumps(redirect_url) # Properly escape the URL
with tempfile.NamedTemporaryFile(mode='w', suffix='.py', delete=False) as wrapper:
wrapper.write(f'''import sys
import subprocess
import os
# Set LEDMATRIX_ROOT
os.environ['LEDMATRIX_ROOT'] = r"{PROJECT_ROOT}"
# Run the auth script and provide redirect URL
proc = subprocess.Popen(
[sys.executable, r"{auth_script}"],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
env=os.environ
)
# Send redirect URL to stdin
redirect_url = {redirect_url_escaped}
stdout, _ = proc.communicate(input=redirect_url + "\\n", timeout=120)
print(stdout)
sys.exit(proc.returncode)
''')
wrapper_path = wrapper.name
try:
result = subprocess.run(
['python3', wrapper_path],
capture_output=True,
text=True,
timeout=120,
env=env
)
if result.returncode == 0:
return jsonify({
'status': 'success',
'message': 'Spotify authentication completed successfully',
'output': result.stdout
})
else:
return jsonify({
'status': 'error',
'message': 'Spotify authentication failed',
'output': result.stdout + result.stderr
}), 400
except subprocess.TimeoutExpired:
return jsonify({'status': 'error', 'message': 'Authentication timed out'}), 408
finally:
# The wrapper carries the user's redirect URL, so it must not
# survive the request on any path — including a failure to
# launch, which the previous per-branch unlinks missed.
if os.path.exists(wrapper_path):
os.unlink(wrapper_path)
else:
# Step 1: Get authorization URL
# Import the script's functions directly to get the auth URL
import sys
import importlib.util
# Load the authentication script as a module
spec = importlib.util.spec_from_file_location("auth_spotify", auth_script)
auth_module = importlib.util.module_from_spec(spec)
sys.modules["auth_spotify"] = auth_module
# Set LEDMATRIX_ROOT before loading
os.environ['LEDMATRIX_ROOT'] = str(PROJECT_ROOT)
try:
spec.loader.exec_module(auth_module)
# Get credentials and create OAuth object
client_id, client_secret, redirect_uri = auth_module.load_spotify_credentials()
if not all([client_id, client_secret, redirect_uri]):
return jsonify({
'status': 'error',
'message': 'Could not load Spotify credentials. Please check config/config_secrets.json.'
}), 400
from spotipy.oauth2 import SpotifyOAuth
sp_oauth = SpotifyOAuth(
client_id=client_id,
client_secret=client_secret,
redirect_uri=redirect_uri,
scope=auth_module.SCOPE,
cache_path=auth_module.SPOTIFY_AUTH_CACHE_PATH,
open_browser=False
)
auth_url = sp_oauth.get_authorize_url()
return jsonify({
'status': 'success',
'message': 'Authorization URL generated',
'auth_url': auth_url
})
except Exception as e:
logger.error("Error getting Spotify auth URL", exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500
except Exception as e:
logger.error('Error in authenticate_spotify', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/authenticate/ytm', methods=['POST'])
def authenticate_ytm():
"""Run YouTube Music authentication script"""
try:
# Get plugin directory
plugin_id = 'ledmatrix-music'
if api_v3.plugin_manager:
plugin_dir = api_v3.plugin_manager.get_plugin_directory(plugin_id)
else:
plugin_dir = PROJECT_ROOT / 'plugins' / plugin_id
if not plugin_dir or not Path(plugin_dir).exists():
return jsonify({'status': 'error', 'message': 'Plugin not found'}), 404
auth_script = Path(plugin_dir) / 'authenticate_ytm.py'
if not auth_script.exists():
return jsonify({'status': 'error', 'message': 'Authentication script not found'}), 404
# Set LEDMATRIX_ROOT environment variable
env = os.environ.copy()
env['LEDMATRIX_ROOT'] = str(PROJECT_ROOT)
# Run the authentication script
result = subprocess.run(
['python3', str(auth_script)],
capture_output=True,
text=True,
timeout=60,
env=env
)
if result.returncode == 0:
return jsonify({
'status': 'success',
'message': 'YouTube Music authentication completed successfully',
'output': result.stdout
})
else:
return jsonify({
'status': 'error',
'message': 'YouTube Music authentication failed',
'output': result.stdout + result.stderr
}), 400
except subprocess.TimeoutExpired:
return jsonify({'status': 'error', 'message': 'Authentication timed out'}), 408
except Exception as e:
logger.error('Error in authenticate_ytm', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
def _plugin_uploads_dir(plugin_id):
"""assets/plugins/<plugin_id>/uploads for a request-supplied id, or None.
@@ -3527,210 +3338,6 @@ def upload_plugin_asset():
'total_files': len(metadata)
})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/of-the-day/json/upload', methods=['POST'])
def upload_of_the_day_json():
"""Upload JSON files for of-the-day plugin"""
try:
if 'files' not in request.files:
return jsonify({'status': 'error', 'message': 'No files provided'}), 400
files = request.files.getlist('files')
if not files or all(not f.filename for f in files):
return jsonify({'status': 'error', 'message': 'No files provided'}), 400
# Get plugin directory
plugin_id = 'ledmatrix-of-the-day'
if api_v3.plugin_manager:
plugin_dir = api_v3.plugin_manager.get_plugin_directory(plugin_id)
else:
plugin_dir = PROJECT_ROOT / 'plugins' / plugin_id
if not plugin_dir or not Path(plugin_dir).exists():
return jsonify({'status': 'error', 'message': 'Plugin not found'}), 404
# Setup of_the_day directory
data_dir = Path(plugin_dir) / 'of_the_day'
data_dir.mkdir(parents=True, exist_ok=True)
uploaded_files = []
max_size_per_file = 5 * 1024 * 1024 # 5MB
for file in files:
if not file.filename:
continue
# Validate file extension
if not file.filename.lower().endswith('.json'):
return jsonify({
'status': 'error',
'message': f'File {file.filename} must be a JSON file (.json)'
}), 400
# Read and validate file size
file.seek(0, os.SEEK_END)
file_size = file.tell()
file.seek(0)
if file_size > max_size_per_file:
return jsonify({
'status': 'error',
'message': f'File {file.filename} exceeds 5MB limit'
}), 400
# Read and validate JSON content
try:
file_content = file.read().decode('utf-8')
json_data = json.loads(file_content)
except json.JSONDecodeError as e:
return jsonify({
'status': 'error',
'message': 'Invalid JSON in request body'
}), 400
except UnicodeDecodeError:
return jsonify({
'status': 'error',
'message': f'File {file.filename} is not valid UTF-8 text'
}), 400
# Validate JSON structure (must be object with day number keys)
if not isinstance(json_data, dict):
return jsonify({
'status': 'error',
'message': f'JSON in {file.filename} must be an object with day numbers (1-365) as keys'
}), 400
# Check if keys are valid day numbers
for key in json_data.keys():
try:
day_num = int(key)
if day_num < 1 or day_num > 365:
return jsonify({
'status': 'error',
'message': f'Day number {day_num} in {file.filename} is out of range (must be 1-365)'
}), 400
except ValueError:
return jsonify({
'status': 'error',
'message': f'Invalid key "{key}" in {file.filename}: must be a day number (1-365)'
}), 400
# Generate safe filename from original (preserve user's filename)
original_filename = file.filename
safe_filename = original_filename.lower().replace(' ', '_')
# Ensure it's a valid filename
safe_filename = ''.join(c for c in safe_filename if c.isalnum() or c in '._-')
if not safe_filename.endswith('.json'):
safe_filename += '.json'
file_path = data_dir / safe_filename
# If file exists, add counter
counter = 1
base_name = safe_filename.replace('.json', '')
while file_path.exists():
safe_filename = f"{base_name}_{counter}.json"
file_path = data_dir / safe_filename
counter += 1
# Save file
with open(file_path, 'w', encoding='utf-8') as f:
json.dump(json_data, f, indent=2, ensure_ascii=False)
# Make file readable
os.chmod(file_path, 0o644)
# Extract category name from filename (remove .json extension)
category_name = safe_filename.replace('.json', '')
display_name = category_name.replace('_', ' ').title()
# Update plugin config to add category
try:
sys.path.insert(0, str(plugin_dir))
from scripts.update_config import add_category_to_config
add_category_to_config(category_name, f'of_the_day/{safe_filename}', display_name)
except Exception as e:
logger.warning("Could not update config: %s", e)
# Continue anyway - file is uploaded
# Generate file ID (use category name as ID for simplicity)
file_id = category_name
uploaded_files.append({
'id': file_id,
'filename': safe_filename,
'original_filename': original_filename,
'path': f'of_the_day/{safe_filename}',
'size': file_size,
'uploaded_at': datetime.utcnow().isoformat() + 'Z',
'category_name': category_name,
'display_name': display_name,
'entry_count': len(json_data)
})
return jsonify({
'status': 'success',
'uploaded_files': uploaded_files,
'total_files': len(uploaded_files)
})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/of-the-day/json/delete', methods=['POST'])
def delete_of_the_day_json():
"""Delete a JSON file from of-the-day plugin"""
try:
data = request.get_json(silent=True) or {}
file_id = data.get('file_id') # This is the category_name
if not file_id:
return jsonify({'status': 'error', 'message': 'file_id is required'}), 400
# file_id names a file that is about to be unlinked, and it arrives
# straight from the request body. Unvalidated, a file_id of
# "../../../../etc/cron" made this endpoint delete any .json file on
# the device the service could write to.
safe_file_id = safe_path_component(file_id)
if not safe_file_id:
return jsonify({'status': 'error', 'message': 'Invalid file_id'}), 400
# Get plugin directory
plugin_id = 'ledmatrix-of-the-day'
if api_v3.plugin_manager:
plugin_dir = api_v3.plugin_manager.get_plugin_directory(plugin_id)
else:
plugin_dir = PROJECT_ROOT / 'plugins' / plugin_id
if not plugin_dir or not Path(plugin_dir).exists():
return jsonify({'status': 'error', 'message': 'Plugin not found'}), 404
filename = f"{safe_file_id}.json"
file_path = resolve_under(Path(plugin_dir) / 'of_the_day', filename)
if file_path is None:
return jsonify({'status': 'error', 'message': 'Invalid file_id'}), 400
if not file_path.exists():
return jsonify({'status': 'error', 'message': f'File {filename} not found'}), 404
# Delete file
file_path.unlink()
# Update config to remove category
try:
sys.path.insert(0, str(plugin_dir))
from scripts.update_config import remove_category_from_config
remove_category_from_config(safe_file_id)
except Exception as e:
logger.warning("Could not update config: %s", e)
return jsonify({
'status': 'success',
'message': f'File {filename} deleted successfully'
})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500