fix(config): Add defaults to schemas and fix config validation issues (#153)

* fix(web): Resolve font display and config API error handling issues

- Fix font catalog display error where path.startsWith fails
  (path is object, not string)
- Update save_main_config to use error_response() helper
- Improve save_raw_main_config error handling consistency
- Add proper error codes and traceback details to API responses

* fix(web): Prevent fontCatalog redeclaration error on HTMX reload

- Use window object to store global font variables
- Check if script has already loaded before declaring variables
- Update both window properties and local references on assignment
- Fixes 'Identifier fontCatalog has already been declared' error

* fix(web): Wrap fonts script in IIFE to prevent all redeclaration errors

- Wrap entire script in IIFE that only runs once
- Check if script already loaded before declaring variables/functions
- Expose initializeFontsTab to window for re-initialization
- Prevents 'Identifier has already been declared' errors on HTMX reload

* fix(web): Exempt config save API endpoints from CSRF protection

- Exempt save_raw_main_config, save_raw_secrets_config, and save_main_config from CSRF
- These endpoints are called via fetch from JavaScript and don't include CSRF tokens
- Fixes 500 error when saving config via raw JSON editor

* fix(web): Exempt system action endpoint from CSRF protection

- Exempt execute_system_action from CSRF
- Fixes 500 error when using system action buttons (restart display, restart Pi, etc.)
- These endpoints are called via HTMX and don't include CSRF tokens

* fix(web): Exempt all API v3 endpoints from CSRF protection

- Add before_request handler to exempt all api_v3.* endpoints
- All API endpoints are programmatic (HTMX/fetch) and don't include CSRF tokens
- Prevents future CSRF errors on any API endpoint
- Cleaner than exempting individual endpoints

* refactor(web): Remove CSRF protection for local-only application

- CSRF is designed for internet-facing apps to prevent cross-site attacks
- For local-only Raspberry Pi app, threat model is different
- All endpoints were exempted anyway, so it wasn't protecting anything
- Forms use HTMX without CSRF tokens
- If exposing to internet later, can re-enable with proper token implementation

* fix(web): Fix font path double-prefixing in font catalog display

- Only prefix with 'assets/fonts/' if path is a bare filename
- If path starts with '/' (absolute) or 'assets/' (already prefixed), use as-is
- Fixes double-prefixing when get_fonts_catalog returns relative paths like 'assets/fonts/press_start.ttf'

* fix(web): Remove fontsTabInitialized guard to allow re-initialization on HTMX reload

- Remove fontsTabInitialized check that prevented re-initialization on HTMX content swap
- The window._fontsScriptLoaded guard is sufficient to prevent function redeclaration
- Allow initializeFontsTab() to run on each HTMX swap to attach listeners to new DOM elements
- Fixes fonts UI breaking after HTMX reload (buttons, upload dropzone, etc. not working)

* fix(api): Preserve empty strings for optional string fields in plugin config

- Add _is_field_required() helper to check if fields are required in schema
- Update _parse_form_value_with_schema() to preserve empty strings for optional string fields
- Fixes 400 error when saving MQTT plugin config with empty username/password
- Resolves validation error: 'Expected type string, got NoneType'

* fix(config): Add defaults to schemas and fix None value handling

- Updated merge_with_defaults to replace None values with defaults
- Fixed form processing to skip empty optional fields without defaults
- Added script to automatically add defaults to all plugin config schemas
- Added defaults to 89 fields across 10 plugin schemas
- Prevents validation errors from None values in configs

Changes:
- schema_manager.py: Enhanced merge_with_defaults to replace None with defaults
- api_v3.py: Added _SKIP_FIELD sentinel to skip optional fields without defaults
- add_defaults_to_schemas.py: Script to add sensible defaults to schemas
- Plugin schemas: Added defaults for number, boolean, and array fields

* fix(config): Fix save button spinner by checking HTTP status code

- Fixed handleConfigSave to check xhr.status instead of event.detail.successful
- With hx-swap="none", HTMX doesn't set event.detail.successful
- Now properly detects successful saves (status 200-299) and stops spinner
- Improved error message extraction from API responses
- Also fixed handleToggleResponse for consistency

---------

Co-authored-by: Chuck <chuck@example.com>

web_interface/blueprints/api_v3.py

@@ -2914,6 +2914,43 @@ def _get_schema_property(schema, key_path):
     return None
 
 
+def _is_field_required(key_path, schema):
+    """
+    Check if a field is required according to the schema.
+    
+    Args:
+        key_path: Dot-separated path like "mqtt.username"
+        schema: The JSON schema dict
+    
+    Returns:
+        True if field is required, False otherwise
+    """
+    if not schema or 'properties' not in schema:
+        return False
+    
+    parts = key_path.split('.')
+    if len(parts) == 1:
+        # Top-level field
+        required = schema.get('required', [])
+        return parts[0] in required
+    else:
+        # Nested field - navigate to parent object
+        parent_path = '.'.join(parts[:-1])
+        field_name = parts[-1]
+        
+        # Get parent property
+        parent_prop = _get_schema_property(schema, parent_path)
+        if not parent_prop or 'properties' not in parent_prop:
+            return False
+        
+        # Check if field is required in parent
+        required = parent_prop.get('required', [])
+        return field_name in required
+
+
+# Sentinel object to indicate a field should be skipped (not set in config)
+_SKIP_FIELD = object()
+
 def _parse_form_value_with_schema(value, key_path, schema):
     """
     Parse a form value using schema information to determine correct type.
@@ -2925,7 +2962,7 @@ def _parse_form_value_with_schema(value, key_path, schema):
         schema: The plugin's JSON schema
     
     Returns:
-        Parsed value with correct type
+        Parsed value with correct type, or _SKIP_FIELD to indicate the field should not be set
     """
     import json
     
@@ -2940,6 +2977,22 @@ def _parse_form_value_with_schema(value, key_path, schema):
         # If schema says it's an object, return empty dict instead of None
         if prop and prop.get('type') == 'object':
             return {}
+        # If it's an optional string field, preserve empty string instead of None
+        if prop and prop.get('type') == 'string':
+            if not _is_field_required(key_path, schema):
+                return ""  # Return empty string for optional string fields
+        # For number/integer fields, check if they have defaults or are required
+        if prop:
+            prop_type = prop.get('type')
+            if prop_type in ('number', 'integer'):
+                # If field has a default, use it
+                if 'default' in prop:
+                    return prop['default']
+                # If field is not required and has no default, skip setting it
+                if not _is_field_required(key_path, schema):
+                    return _SKIP_FIELD
+                # If field is required but empty, return None (validation will fail, which is correct)
+                return None
         return None
     
     # Handle string values
@@ -3029,8 +3082,12 @@ def _set_nested_value(config, key_path, value):
     Args:
         config: The config dict to modify
         key_path: Dot-separated path (e.g., "customization.period_text.font")
-        value: The value to set
+        value: The value to set (or _SKIP_FIELD to skip setting)
     """
+    # Skip setting if value is the sentinel
+    if value is _SKIP_FIELD:
+        return
+    
     parts = key_path.split('.')
     current = config
     
@@ -3231,7 +3288,9 @@ def save_plugin_config():
                 import logging
                 logger = logging.getLogger(__name__)
                 logger.debug(f"Combined indexed array field {base_path}: {values} -> {combined_value} -> {parsed_value}")
-                _set_nested_value(plugin_config, base_path, parsed_value)
+                # Only set if not skipped
+                if parsed_value is not _SKIP_FIELD:
+                    _set_nested_value(plugin_config, base_path, parsed_value)
             
             # Process remaining (non-indexed) fields
             # Skip any base paths that were processed as indexed arrays
@@ -3249,8 +3308,9 @@ def save_plugin_config():
                                 import logging
                                 logger = logging.getLogger(__name__)
                                 logger.debug(f"Array field {key}: form value='{value}' -> parsed={parsed_value}")
-                        # Use helper to set nested values correctly
-                        _set_nested_value(plugin_config, key, parsed_value)
+                        # Use helper to set nested values correctly (skips if _SKIP_FIELD)
+                        if parsed_value is not _SKIP_FIELD:
+                            _set_nested_value(plugin_config, key, parsed_value)
             
             # Post-process: Fix array fields that might have been incorrectly structured
             # This handles cases where array fields are stored as dicts (e.g., from indexed form fields)