LEDMatrix/docs/archive/CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md

Implementation Plan: Fix Config Schema Validation Issues

Based on audit results showing 186 issues across 20 plugins.

Overview

Three priority fixes identified from audit:

1. Priority 1 (HIGH): Remove core properties from required array - will fix ~150 issues
2. Priority 2 (MEDIUM): Verify default merging logic - will fix remaining required field issues
3. Priority 3 (LOW): Calendar plugin schema cleanup - will fix 3 extra field warnings

Priority 1: Remove Core Properties from Required Array

Problem

Core properties (enabled, display_duration, live_priority) are system-managed but listed in schema required arrays. SchemaManager injects them into properties but doesn't remove them from required, causing validation failures.

Solution

File: src/plugin_system/schema_manager.py Location: validate_config_against_schema() method, after line 295

Implementation Steps

1. Add code to remove core properties from required array:

   # After injecting core properties (around line 295), add:
   # Remove core properties from required array (they're system-managed)
   if "required" in enhanced_schema:
       core_prop_names = list(core_properties.keys())
       enhanced_schema["required"] = [
           field for field in enhanced_schema["required"] 
           if field not in core_prop_names
       ]
   

2. Add logging for debugging (optional but helpful):

   if "required" in enhanced_schema and core_prop_names:
       removed_from_required = [
           field for field in enhanced_schema.get("required", [])
           if field in core_prop_names
       ]
       if removed_from_required and plugin_id:
           self.logger.debug(
               f"Removed core properties from required array for {plugin_id}: {removed_from_required}"
           )
   

3. Test the fix:

   • Run audit script: python scripts/audit_plugin_configs.py
   • Expected: Issue count drops from 186 to ~30-40
   • All "enabled" related errors should be eliminated

Expected Outcome

• All 20 plugins should no longer fail validation due to missing enabled field
• ~150 issues resolved (all enabled-related validation errors)

Priority 2: Verify Default Merging Logic

Problem

Some plugins have required fields with defaults that should be applied before validation. Need to verify the default merging happens correctly and handles nested objects.

Solution

File: web_interface/blueprints/api_v3.py Location: save_plugin_config() method, around lines 3218-3221

Implementation Steps

1. Review current default merging logic:

   • Check that merge_with_defaults() is called before validation (line 3220)
   • Verify it's called after preserving enabled state but before validation

2. Verify merge_with_defaults handles nested objects:

   • Check src/plugin_system/schema_manager.py → merge_with_defaults() method
   • Ensure it recursively merges nested objects (it does use deep_merge)
   • Test with plugins that have nested required fields

3. Check if defaults are applied for nested required fields:

   • Review how generate_default_config() extracts defaults from nested schemas
   • Verify nested required fields with defaults are included

4. Test with problematic plugins:

   • ledmatrix-weather: required fields api_key, location_city (check if defaults exist)
   • mqtt-notifications: required field mqtt object (check if default exists)
   • text-display: required field text (check if default exists)
   • ledmatrix-music: required field preferred_source (check if default exists)

5. If defaults don't exist in schemas:

   • Either add defaults to schemas, OR
   • Make fields optional in schemas if they're truly optional

Expected Outcome

• Plugins with required fields that have schema defaults should pass validation
• Issue count further reduced from ~30-40 to ~5-10

Priority 3: Calendar Plugin Schema Cleanup

Problem

Calendar plugin config has fields not in schema:

• show_all_day (config) but schema has show_all_day_events (field name mismatch)
• date_format (not in schema, not used in manager.py)
• time_format (not in schema, not used in manager.py)

Investigation Results

• Schema defines: show_all_day_events (boolean, default: true)
• Manager.py uses: show_all_day_events (line 82: config.get('show_all_day_events', True))
• Config has: show_all_day (wrong field name - should be show_all_day_events)
• date_format and time_format appear to be deprecated (not used in manager.py)

Solution

File: config/config.json → calendar section

Implementation Steps

1. Fix field name mismatch:

   • Rename show_all_day → show_all_day_events in config.json
   • This matches the schema and manager.py code

2. Remove deprecated fields:

   • Remove date_format from config (not used in code)
   • Remove time_format from config (not used in code)

3. Alternative (if fields are needed): Add date_format and time_format to schema

   • Only if these fields should be supported
   • Check if they're used anywhere else in the codebase

4. Test calendar plugin:

   • Run audit for calendar plugin specifically
   • Verify no extra field warnings remain
   • Test calendar plugin functionality to ensure it still works

Expected Outcome

• Calendar plugin shows 0 extra field warnings
• Final issue count: ~3-5 (only edge cases remain)

Testing Strategy

After Each Priority Fix

1. Run local audit:

   python scripts/audit_plugin_configs.py
   

2. Check issue count reduction:

   • Priority 1: Should drop from 186 to ~30-40
   • Priority 2: Should drop from ~30-40 to ~5-10
   • Priority 3: Should drop from ~5-10 to ~3-5

3. Review specific plugin results:

   python scripts/audit_plugin_configs.py --plugin <plugin-id>
   

After All Fixes

1. Full audit run:

   python scripts/audit_plugin_configs.py
   

2. Deploy to Pi:

   ./scripts/deploy_to_pi.sh src/plugin_system/schema_manager.py web_interface/blueprints/api_v3.py
   

3. Run audit on Pi:

   ./scripts/run_audit_on_pi.sh
   

4. Manual web interface testing:

   • Access each problematic plugin's config page
   • Try saving configuration
   • Verify no validation errors appear
   • Check that configs save successfully

Success Criteria

• Priority 1: All "enabled" related validation errors eliminated
• Priority 1: Issue count reduced from 186 to ~30-40
• Priority 2: Plugins with required fields + defaults pass validation
• Priority 2: Issue count reduced to ~5-10
• Priority 3: Calendar plugin extra field warnings resolved
• Priority 3: Final issue count at ~3-5 (only edge cases)
• All fixes work on Pi (not just local)
• Web interface saves configs without validation errors

Files to Modify

1. src/plugin_system/schema_manager.py - Remove core properties from required array
2. plugins/calendar/config_schema.json OR config/config.json - Calendar cleanup (if needed)
3. web_interface/blueprints/api_v3.py - May need minor adjustments for default merging (if needed)

Risk Assessment

Priority 1: Low risk - Only affects validation logic, doesn't change behavior Priority 2: Low risk - Only ensures defaults are applied (already intended behavior) Priority 3: Very low risk - Only affects calendar plugin, cosmetic issue

All changes are backward compatible and improve the system rather than changing core functionality.