Files
LEDMatrix/web_interface/blueprints/api_v3/backup.py
T
ChuckandClaude Opus 5.5 a8b3e86775 refactor(web): delete dead routes, JS files and duplicate definitions (#609)
* refactor(web): drop validators nothing calls

escape_html, validate_image_url, validate_font_awesome_class,
validate_mime_type, validate_numeric_range, validate_string_length and
sanitize_plugin_config had no callers outside their own tests. Only
validate_file_upload (fonts upload) is imported by the web interface.

dedup_unique_arrays is kept: its one caller in save_plugin_config was
removed by the unrelated sync PR (#330), which looks accidental.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(api): remove the music-auth and of-the-day JSON routes

POST /plugins/authenticate/spotify and /plugins/authenticate/ytm had no
caller but their tests: the music plugin authenticates through its
web_ui_actions (authenticate_spotify.py / authenticate_ytm.py) via
/plugins/action.

POST /plugins/of-the-day/json/upload and /json/delete looked the plugin
up by the id ledmatrix-of-the-day (its manifest id is of-the-day), were
reachable only from a file_type "json" upload field that no schema
declares, and put the plugin directory on sys.path per request to
import scripts.update_config. of-the-day manages its files through
plugin-file-manager and its own web_ui_actions.

The of-the-day branch of GET /plugins/config stays: it matches the real
manifest id and still merges the on-disk category files into the form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(api): read managers only from the blueprints

api_v3/__init__.py and pages_v3.py declared module globals
(plugin_store_manager, saved_repositories_manager, schema_manager,
operation_queue, plugin_state_manager, operation_history, sync_manager,
config_manager, plugin_manager) that nothing assigns: app.py sets the
managers as attributes on the Blueprint objects, and every route reads
them there. The one reader, backup restore's fallback to the module
plugin_store_manager, could only ever fall back to None.

_ensure_cache_manager() built a second CacheManager in the web process
instead of using the one app.py puts on api_v3. The display routes now
read api_v3.cache_manager, creating it on the blueprint only when
nothing set it (the same None handling as the /cache routes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(web): drop run.sh and the unused log_config_change

web_interface/run.sh was referenced only by web_interface/README.md;
the service starts the UI through scripts/utils/start_web_conditionally.py
and the README already documents `python3 web_interface/start.py`.
log_config_change() in web_interface/logging_config.py was never called.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): delete unreferenced store_manager.js, diff_viewer.js, htmx-sse.js

- js/plugins/store_manager.js (window.PluginStoreManager) and
  js/config/diff_viewer.js (window.ConfigDiffViewer) were loaded on every
  page but nothing reads either global.
- htmx-sse.js (plus its CDN fallback) was loaded after HTMX, but no
  template or plugin page uses sse-connect / hx-ext="sse": the live
  streams run through LEDStreams in app-shell.js.

js/plugins/state_manager.js stays: install_manager.js's updateAll()
reads and refreshes window.PluginStateManager.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): remove app.js helpers nothing calls

- hexToRgb, rgbToHex, validateForm, uploadFont and switchTab (whose
  'switch-tab' event had no listener) have no caller in the templates,
  static JS or the plugin monorepo.
- installPlugin: plugins_manager.js (loaded last) assigns
  window.installPlugin, and its own store cards are the only callers.
- The showNotification fallback could never install: app-shell.js is
  deferred ahead of app.js and defines the same fallback at top level.
- performanceMonitor only logged with ?debug=perf and read an unset
  this.measures; the marks it took on every load had no reader.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): drop app-shell.js refreshPlugin

A top-level function in app-shell.js, so a window global, but nothing
calls it (no inline handler, no window lookup, no string-built name).

The other plugin actions in that block stay. updatePlugin is the live
window.updatePlugin: plugins_manager.js only installs its own copy when
none exists. uninstallPlugin/pollUninstallOperation, updateAllPlugins,
executePluginAction and toggleNestedSection are replaced by later
deferred scripts, but a click that lands while those scripts are still
downloading reaches the app-shell copies, so removing them is not a
pure no-op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): remove definitions plugins_manager.js always overrides

All of these are replaced before anything can call them, checked
against the load order in base.html and the live window.* values:

- openOnDemandModal/requestOnDemandStop stubs: the IIFE later in the
  same script assigns the real functions synchronously.
- updatePlugin and uninstallPlugin stubs (`window.X || stub`): app-shell.js
  already defined both, so the fallback never installed. Same for the
  later updatePlugin override, gated on the live function containing
  '[UPDATE]', which app-shell.js's never does.
- The first addArrayObjectItem/removeArrayObjectItem: reassigned by the
  top-level copies after the IIFE.
- The first `function formatDate` in the IIFE: a later declaration of
  the same name in the same scope wins.
- deleteUploadedImage, getCurrentImages, showUploadProgress,
  formatFileSize and getScheduleSummary: character-for-character
  copies of js/widgets/file-upload.js, which stays the owner.
- `typeof X === 'undefined'` fallbacks and `typeof X !== 'undefined'`
  re-exports after the IIFE: always false, or a self-assignment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): render the shell directly and delete index.html

index.html extended base.html with {% block content %}, but base.html
defines no blocks, so none of index.html ever rendered: rendering both
with jinja2 gives byte-identical output. index() still loaded the config,
read config.json and config_secrets.json raw and json.dumps'd them on
every page load for variables base.html never reads, and flashed errors
that base.html never shows. It now renders base.html with no context.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): stop htmx-config.js replacing console.error and console.warn

It swapped both globals for filters that dropped any error mentioning
insertBefore / "Cannot read properties of null" when "htmx" appeared in
the message or stack, and a list of Permissions-Policy warnings. That
hid real errors from every script on the page, and made every logged
error and warning report htmx-config.js as its source. The beforeSwap
target validation above it, which prevents the insertBefore errors in
the first place, stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(web): quiet the widget load announcements and debug logs

About 30 lines hit the console on every page load: one "... widget
registered" per widget file, one "[WidgetRegistry] Registered widget: X"
per registration, plus the registry, base widget and plugin loader
announcing themselves. The load-time announcements are removed; the
per-call ones (registry register, plugin widget loads, "Render called")
now go through the page's debugLog switch (localStorage.pluginDebug),
guarded because the widgets also load in node tests without it.
fonts.html and wifi.html debug logging goes through debugLog as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(api): drop the removed music-auth and of-the-day JSON routes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 12:36:53 -04:00

230 lines
11 KiB
Python

"""Backup creation, listing and restore.
Routes decorate the shared `api_v3` Blueprint from ._common, so their
endpoint names are unchanged by living here.
"""
from web_interface.blueprints.api_v3 import (
PROJECT_ROOT, Path, _coerce_to_bool, _safe_backup_path, api_v3,
datetime, json, jsonify, logger, os, request, tempfile,
)
import web_interface.blueprints.api_v3 as _pkg
# Read through the module rather than bound by value: tests patch these
# as module attributes, and a value binding would not see the patch.
# Several are also called from helpers that live in __init__, so the
# package is the only patch point that covers every caller.
@api_v3.route('/backup/preview', methods=['GET'])
def backup_preview():
"""Return a summary of what a new backup would include."""
try:
from src.backup_manager import preview_backup_contents
data = preview_backup_contents(PROJECT_ROOT)
return jsonify({'status': 'success', 'data': data})
except Exception as e:
logger.error("backup_preview failed: %s", e, exc_info=True)
return jsonify({'status': 'error', 'message': 'An internal error occurred; see logs for details'}), 500
@api_v3.route('/backup/list', methods=['GET'])
def backup_list():
"""List backup ZIPs stored in the export directory."""
try:
_pkg._BACKUP_EXPORT_DIR.mkdir(parents=True, exist_ok=True)
entries = []
for p in sorted(_pkg._BACKUP_EXPORT_DIR.iterdir(), key=lambda x: x.stat().st_mtime, reverse=True):
if not p.is_file() or p.suffix != '.zip':
continue
st = p.stat()
entries.append({
'filename': p.name,
'size': st.st_size,
'created_at': datetime.fromtimestamp(st.st_mtime).strftime('%Y-%m-%d %H:%M:%S'),
})
return jsonify({'status': 'success', 'data': entries})
except Exception as e:
logger.error("backup_list failed: %s", e, exc_info=True)
return jsonify({'status': 'error', 'message': 'An internal error occurred; see logs for details'}), 500
@api_v3.route('/backup/export', methods=['POST'])
def backup_export():
"""Create a new backup ZIP and return its filename."""
try:
from src.backup_manager import create_backup
zip_path = create_backup(PROJECT_ROOT, output_dir=_pkg._BACKUP_EXPORT_DIR)
return jsonify({'status': 'success', 'filename': zip_path.name})
except Exception as e:
logger.error("backup_export failed: %s", e, exc_info=True)
return jsonify({'status': 'error', 'message': 'An internal error occurred; see logs for details'}), 500
@api_v3.route('/backup/validate', methods=['POST'])
def backup_validate():
"""Validate an uploaded backup ZIP and return its manifest."""
try:
from src.backup_manager import validate_backup
if 'backup_file' not in request.files:
return jsonify({'status': 'error', 'message': 'No backup_file in request'}), 400
f = request.files['backup_file']
with tempfile.NamedTemporaryFile(suffix='.zip', delete=False) as tmp:
tmp_path = tmp.name
f.save(tmp_path)
try:
ok, err_msg, manifest = validate_backup(Path(tmp_path))
finally:
try:
os.unlink(tmp_path)
except OSError:
pass
if not ok:
logger.warning("Backup validation failed: %s", err_msg)
return jsonify({'status': 'error', 'message': 'Invalid or corrupted backup file'}), 400
safe_manifest = {
'schema_version': manifest.get('schema_version'),
'created_at': manifest.get('created_at'),
'ledmatrix_version': manifest.get('ledmatrix_version'),
'hostname': manifest.get('hostname'),
'contents': manifest.get('contents', []),
'detected_contents': manifest.get('detected_contents', []),
'plugins': manifest.get('plugins', []),
'total_uncompressed': manifest.get('total_uncompressed'),
'file_count': manifest.get('file_count'),
}
return jsonify({'status': 'success', 'data': safe_manifest})
except Exception as e:
logger.error("backup_validate failed: %s", e, exc_info=True)
return jsonify({'status': 'error', 'message': 'An internal error occurred; see logs for details'}), 500
#: The only keys RestoreOptions recognizes. A typo'd or renamed key (e.g.
#: "restoreSecrets") would otherwise be silently ignored by opts_dict.get(),
#: leaving that flag at its True default -- restoring secrets a caller's
#: request clearly meant to exclude, with no indication anything was wrong.
_RESTORE_OPTION_KEYS = frozenset((
'restore_config', 'restore_secrets', 'restore_wifi', 'restore_fonts',
'restore_plugin_uploads', 'reinstall_plugins',
))
@api_v3.route('/backup/restore', methods=['POST'])
def backup_restore():
"""Restore a backup ZIP with optional RestoreOptions."""
try:
from src.backup_manager import restore_backup, RestoreOptions
if 'backup_file' not in request.files:
return jsonify({'status': 'error', 'message': 'No backup_file in request'}), 400
f = request.files['backup_file']
options_raw = request.form.get('options', '{}')
try:
opts_dict = json.loads(options_raw)
except json.JSONDecodeError:
opts_dict = None
if not isinstance(opts_dict, dict):
# Every option defaults to True, so falling back to {} on a
# parse failure would silently perform a FULL restore —
# secrets and all — for a caller who asked for a narrow one
# and mis-serialized it. Refuse instead of guessing.
return jsonify({
'status': 'error',
'message': 'Invalid options: expected a JSON object',
}), 400
unknown_keys = set(opts_dict) - _RESTORE_OPTION_KEYS
if unknown_keys:
return jsonify({
'status': 'error',
'message': f'Unknown restore option(s): {", ".join(sorted(unknown_keys))}',
}), 400
# _coerce_to_bool (not bare bool()) because a request can send these
# as JSON strings: bool("false") is True in Python, so a caller who
# explicitly asked to skip secrets would have had them restored
# anyway.
options = RestoreOptions(
restore_config=_coerce_to_bool(opts_dict.get('restore_config', True)),
restore_secrets=_coerce_to_bool(opts_dict.get('restore_secrets', True)),
restore_wifi=_coerce_to_bool(opts_dict.get('restore_wifi', True)),
restore_fonts=_coerce_to_bool(opts_dict.get('restore_fonts', True)),
restore_plugin_uploads=_coerce_to_bool(opts_dict.get('restore_plugin_uploads', True)),
reinstall_plugins=_coerce_to_bool(opts_dict.get('reinstall_plugins', True)),
)
with tempfile.NamedTemporaryFile(suffix='.zip', delete=False) as tmp:
tmp_path = tmp.name
f.save(tmp_path)
try:
result = restore_backup(Path(tmp_path), PROJECT_ROOT, options)
finally:
try:
os.unlink(tmp_path)
except OSError:
pass
# Reinstall plugins if requested and store manager available
if options.reinstall_plugins and result.plugins_to_install:
psm = getattr(api_v3, 'plugin_store_manager', None)
for plug in result.plugins_to_install:
pid = plug.get('plugin_id')
if not pid:
continue
try:
if psm and hasattr(psm, 'install_plugin'):
ok = psm.install_plugin(pid)
if ok:
result.plugins_installed.append(pid)
else:
result.plugins_failed.append({'plugin_id': pid, 'error': 'install_plugin returned False'})
else:
result.plugins_failed.append({'plugin_id': pid, 'error': 'Store manager unavailable'})
except Exception as pe:
logger.error(
"[Backup] Failed to reinstall plugin %r: %s", pid, pe, exc_info=True
)
result.plugins_failed.append({'plugin_id': pid, 'error': 'Installation failed; see server logs'})
# A restore that dropped files can still report success if the only
# failures were plugin reinstalls, since those don't touch result.errors.
if result.plugins_failed:
result.success = False
data = result.to_dict()
if not result.success:
# Name what failed, and what nonetheless landed. A restore is
# partial far more often than it is total -- a fresh install can
# leave config_secrets.json unwritable by the web service, so
# config restores and secrets do not. "Restore had errors" alone
# left the user unable to tell a wholly failed restore from one
# that quietly dropped their API keys.
failed_plugins = [
str(p.get('plugin_id')) for p in (result.plugins_failed or []) if p.get('plugin_id')
]
parts = []
if result.restored:
parts.append(f"restored: {', '.join(result.restored)}")
if result.errors:
parts.append(f"failed: {'; '.join(result.errors)}")
if failed_plugins:
parts.append(f"plugins not reinstalled: {', '.join(failed_plugins)}")
message = 'Restore incomplete — ' + ('. '.join(parts) if parts else 'see logs')
return jsonify({'status': 'error', 'message': message, 'data': data}), 500
return jsonify({'status': 'success', 'data': data})
except Exception as e:
logger.error("backup_restore failed: %s", e, exc_info=True)
return jsonify({'status': 'error', 'message': 'An internal error occurred; see logs for details'}), 500
@api_v3.route('/backup/download/<path:filename>', methods=['GET'])
def backup_download(filename):
"""Stream a backup ZIP to the browser."""
from flask import send_from_directory
if _safe_backup_path(filename) is None:
return jsonify({'status': 'error', 'message': 'Backup not found'}), 404
try:
# send_from_directory uses werkzeug safe_join internally — CodeQL-recognized sanitizer.
return send_from_directory(_pkg._BACKUP_EXPORT_DIR, filename, as_attachment=True)
except FileNotFoundError:
return jsonify({'status': 'error', 'message': 'Backup not found'}), 404
@api_v3.route('/backup/<path:filename>', methods=['DELETE'])
def backup_delete(filename):
"""Delete a stored backup ZIP."""
safe = _safe_backup_path(filename)
if safe is None:
return jsonify({'status': 'error', 'message': 'Backup not found'}), 404
# Enumerate the export directory and match by name so the unlink target is
# a filesystem-derived path rather than one constructed from user input.
try:
for entry in _pkg._BACKUP_EXPORT_DIR.iterdir():
if entry.is_file() and entry.name == safe.name:
entry.unlink()
return jsonify({'status': 'success'})
except OSError as e:
logger.error("backup_delete failed: %s", e, exc_info=True)
return jsonify({'status': 'error', 'message': 'An internal error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'Backup not found'}), 404