Files
LEDMatrix/scripts/add_defaults_to_schemas.py
Chuck 7ec4323ff4 fix(config): Add defaults to schemas and fix config validation issues (#153)
* fix(web): Resolve font display and config API error handling issues

- Fix font catalog display error where path.startsWith fails
  (path is object, not string)
- Update save_main_config to use error_response() helper
- Improve save_raw_main_config error handling consistency
- Add proper error codes and traceback details to API responses

* fix(web): Prevent fontCatalog redeclaration error on HTMX reload

- Use window object to store global font variables
- Check if script has already loaded before declaring variables
- Update both window properties and local references on assignment
- Fixes 'Identifier fontCatalog has already been declared' error

* fix(web): Wrap fonts script in IIFE to prevent all redeclaration errors

- Wrap entire script in IIFE that only runs once
- Check if script already loaded before declaring variables/functions
- Expose initializeFontsTab to window for re-initialization
- Prevents 'Identifier has already been declared' errors on HTMX reload

* fix(web): Exempt config save API endpoints from CSRF protection

- Exempt save_raw_main_config, save_raw_secrets_config, and save_main_config from CSRF
- These endpoints are called via fetch from JavaScript and don't include CSRF tokens
- Fixes 500 error when saving config via raw JSON editor

* fix(web): Exempt system action endpoint from CSRF protection

- Exempt execute_system_action from CSRF
- Fixes 500 error when using system action buttons (restart display, restart Pi, etc.)
- These endpoints are called via HTMX and don't include CSRF tokens

* fix(web): Exempt all API v3 endpoints from CSRF protection

- Add before_request handler to exempt all api_v3.* endpoints
- All API endpoints are programmatic (HTMX/fetch) and don't include CSRF tokens
- Prevents future CSRF errors on any API endpoint
- Cleaner than exempting individual endpoints

* refactor(web): Remove CSRF protection for local-only application

- CSRF is designed for internet-facing apps to prevent cross-site attacks
- For local-only Raspberry Pi app, threat model is different
- All endpoints were exempted anyway, so it wasn't protecting anything
- Forms use HTMX without CSRF tokens
- If exposing to internet later, can re-enable with proper token implementation

* fix(web): Fix font path double-prefixing in font catalog display

- Only prefix with 'assets/fonts/' if path is a bare filename
- If path starts with '/' (absolute) or 'assets/' (already prefixed), use as-is
- Fixes double-prefixing when get_fonts_catalog returns relative paths like 'assets/fonts/press_start.ttf'

* fix(web): Remove fontsTabInitialized guard to allow re-initialization on HTMX reload

- Remove fontsTabInitialized check that prevented re-initialization on HTMX content swap
- The window._fontsScriptLoaded guard is sufficient to prevent function redeclaration
- Allow initializeFontsTab() to run on each HTMX swap to attach listeners to new DOM elements
- Fixes fonts UI breaking after HTMX reload (buttons, upload dropzone, etc. not working)

* fix(api): Preserve empty strings for optional string fields in plugin config

- Add _is_field_required() helper to check if fields are required in schema
- Update _parse_form_value_with_schema() to preserve empty strings for optional string fields
- Fixes 400 error when saving MQTT plugin config with empty username/password
- Resolves validation error: 'Expected type string, got NoneType'

* fix(config): Add defaults to schemas and fix None value handling

- Updated merge_with_defaults to replace None values with defaults
- Fixed form processing to skip empty optional fields without defaults
- Added script to automatically add defaults to all plugin config schemas
- Added defaults to 89 fields across 10 plugin schemas
- Prevents validation errors from None values in configs

Changes:
- schema_manager.py: Enhanced merge_with_defaults to replace None with defaults
- api_v3.py: Added _SKIP_FIELD sentinel to skip optional fields without defaults
- add_defaults_to_schemas.py: Script to add sensible defaults to schemas
- Plugin schemas: Added defaults for number, boolean, and array fields

* fix(config): Fix save button spinner by checking HTTP status code

- Fixed handleConfigSave to check xhr.status instead of event.detail.successful
- With hx-swap="none", HTMX doesn't set event.detail.successful
- Now properly detects successful saves (status 200-299) and stops spinner
- Improved error message extraction from API responses
- Also fixed handleToggleResponse for consistency

---------

Co-authored-by: Chuck <chuck@example.com>
2025-12-28 09:18:20 -05:00

232 lines
6.8 KiB
Python
Executable File

#!/usr/bin/env python3
"""
Script to add default values to plugin config schemas where missing.
This ensures that configs never start with None values, improving user experience
and preventing validation errors.
"""
import json
import sys
from pathlib import Path
from typing import Any, Dict, List, Optional
def get_default_for_field(prop: Dict[str, Any]) -> Any:
"""
Determine a sensible default value for a field based on its type and constraints.
Args:
prop: Field property schema
Returns:
Default value or None if no default should be added
"""
prop_type = prop.get('type')
# Handle union types (array with multiple types)
if isinstance(prop_type, list):
# Use the first non-null type
prop_type = next((t for t in prop_type if t != 'null'), prop_type[0] if prop_type else 'string')
if prop_type == 'boolean':
return False
elif prop_type == 'number':
# For numbers, use minimum if available, or a sensible default
minimum = prop.get('minimum')
maximum = prop.get('maximum')
if minimum is not None:
return minimum
elif maximum is not None:
# Use a reasonable fraction of max (like 30% or minimum 1)
return max(1, int(maximum * 0.3))
else:
# No constraints, use 0
return 0
elif prop_type == 'integer':
# Similar to number
minimum = prop.get('minimum')
maximum = prop.get('maximum')
if minimum is not None:
return minimum
elif maximum is not None:
return max(1, int(maximum * 0.3))
else:
return 0
elif prop_type == 'string':
# Only add default for strings if it makes sense
# Check if there's an enum - use first value
enum_values = prop.get('enum')
if enum_values:
return enum_values[0]
# For optional string fields, empty string might be okay, but be cautious
# We'll skip adding defaults for strings unless explicitly needed
return None
elif prop_type == 'array':
# Empty array as default
return []
elif prop_type == 'object':
# Empty object - but we'll handle nested objects separately
return {}
return None
def should_add_default(prop: Dict[str, Any], field_path: str) -> bool:
"""
Determine if we should add a default value to this field.
Args:
prop: Field property schema
field_path: Dot-separated path to the field
Returns:
True if default should be added
"""
# Skip if already has a default
if 'default' in prop:
return False
# Skip secret fields (they should be user-provided)
if prop.get('x-secret', False):
return False
# Skip API keys and similar sensitive fields
field_name = field_path.split('.')[-1].lower()
sensitive_keywords = ['key', 'password', 'secret', 'token', 'auth', 'credential']
if any(keyword in field_name for keyword in sensitive_keywords):
return False
prop_type = prop.get('type')
if isinstance(prop_type, list):
prop_type = next((t for t in prop_type if t != 'null'), prop_type[0] if prop_type else None)
# Only add defaults for certain types
if prop_type in ('boolean', 'number', 'integer', 'array'):
return True
# For strings, only if there's an enum
if prop_type == 'string' and 'enum' in prop:
return True
return False
def add_defaults_recursive(schema: Dict[str, Any], path: str = "", modified: List[str] = None) -> bool:
"""
Recursively add default values to schema fields.
Args:
schema: Schema dictionary to modify
path: Current path in the schema (for logging)
modified: List to track which fields were modified
Returns:
True if any modifications were made
"""
if modified is None:
modified = []
if not isinstance(schema, dict) or 'properties' not in schema:
return False
changes_made = False
for key, prop in schema['properties'].items():
if not isinstance(prop, dict):
continue
current_path = f"{path}.{key}" if path else key
# Check nested objects
if prop.get('type') == 'object' and 'properties' in prop:
if add_defaults_recursive(prop, current_path, modified):
changes_made = True
# Add default if appropriate
if should_add_default(prop, current_path):
default_value = get_default_for_field(prop)
if default_value is not None:
prop['default'] = default_value
modified.append(current_path)
changes_made = True
print(f" Added default to {current_path}: {default_value} (type: {prop.get('type')})")
return changes_made
def process_schema_file(schema_path: Path) -> bool:
"""
Process a single schema file to add defaults.
Args:
schema_path: Path to the schema file
Returns:
True if file was modified
"""
print(f"\nProcessing: {schema_path}")
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
except Exception as e:
print(f" Error reading schema: {e}")
return False
modified_fields = []
changes_made = add_defaults_recursive(schema, modified=modified_fields)
if changes_made:
# Write back with pretty formatting
with open(schema_path, 'w', encoding='utf-8') as f:
json.dump(schema, f, indent=2, ensure_ascii=False)
f.write('\n') # Add trailing newline
print(f" ✓ Modified {len(modified_fields)} fields")
return True
else:
print(f" ✓ No changes needed")
return False
def main():
"""Main entry point."""
project_root = Path(__file__).parent.parent
plugins_dir = project_root / 'plugins'
if not plugins_dir.exists():
print(f"Error: Plugins directory not found: {plugins_dir}")
sys.exit(1)
# Find all config_schema.json files
schema_files = list(plugins_dir.rglob('config_schema.json'))
if not schema_files:
print("No config_schema.json files found")
sys.exit(0)
print(f"Found {len(schema_files)} schema files")
modified_count = 0
for schema_file in sorted(schema_files):
if process_schema_file(schema_file):
modified_count += 1
print(f"\n{'='*60}")
print(f"Summary: Modified {modified_count} out of {len(schema_files)} schema files")
print(f"{'='*60}")
if __name__ == '__main__':
main()