fix(plugins): stop reconciliation inventing plugins and telling users to delete real config (#557)

On a device running four installed, configured, working plugins, the overview
banner read:

  Stale plugin config entries found: football-scoreboard, odds-ticker, data,
  ledmatrix-weather, starlark-apps. Remove them from config.json or reinstall
  via the Plugin Store.

Every claim in that sentence was wrong, and following its advice would have
deleted 4.9KB of working league settings. Four separate defects combined.

1. Secrets keys became phantom plugins. load_config() merges
   config_secrets.json into the config it returns, and the ignore list named
   only 'github' and 'youtube'. A 'data' key in that file therefore read as a
   plugin id and was reported as "in config but not on disk" forever. Read the
   secrets file's own top-level keys instead of hardcoding two of them.

2. The auto-fix clobbered real config. The handler for "on disk but not in
   config" assigned `config[plugin_id] = {'enabled': False}` unconditionally,
   so whenever detection was wrong it replaced a plugin's entire configuration
   with a stub. On the reported device it only failed to do so because the
   write hit EACCES. Now it refuses to overwrite an entry that already exists.

3. The banner gave backwards advice. plugin_missing_in_config ("on disk, not in
   config") and plugin_missing_on_disk ("in config, not on disk") are opposite
   problems, and both were rendered as "stale config entries ... remove them
   from config.json" -- which is correct for the second and destructive for the
   first. They are now reported separately, each with the advice that fits.

4. A stale verdict was served indefinitely. The result is a snapshot written
   once per run to a status file, and a run that fails to apply a fix also
   declares it will not retry. A condition that had since resolved kept being
   reported for hours. The status endpoint now re-checks stored findings
   against current state, dropping only what it can prove stale and keeping
   any kind it cannot re-verify.

The secrets-key lookup is deliberately fail-safe: an unreadable, absent,
malformed or non-path secrets location narrows the ignore set rather than
raising. An earlier revision let TypeError escape, which the broad handler in
_get_config_state() swallowed as "Error reading config state" -- emptying the
config state and making every downstream detection wrong. The existing
reconciliation tests caught it; there is now a regression test for it too.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

web_interface/blueprints/api_v3.py

@@ -3464,6 +3464,30 @@ def reconcile_plugin_state():
             status_code=500
         )
 
+def _drop_stale_reconciliation_findings(unresolved):
+    """Re-check a stored reconciliation verdict against current state.
+
+    The verdict is a snapshot written once per run, and a run that could not
+    apply a fix also refuses to retry -- so a resolved condition was reported
+    indefinitely. Best-effort: any failure here returns the list untouched,
+    because showing a stale warning beats failing the endpoint.
+    """
+    try:
+        from src.plugin_system.state_reconciliation import still_unresolved
+
+        config_keys = set(api_v3.config_manager.load_config() or {})
+        installed = set()
+        plugins_dir = getattr(api_v3.plugin_manager, 'plugins_dir', None)
+        if plugins_dir:
+            for entry in Path(plugins_dir).iterdir():
+                if entry.is_dir() and (entry / 'manifest.json').exists():
+                    installed.add(entry.name)
+        return still_unresolved(unresolved, config_keys, installed)
+    except Exception:
+        logger.debug("[Reconciliation] Could not re-check stored findings", exc_info=True)
+        return unresolved
+
+
 @api_v3.route('/plugins/reconciliation-status', methods=['GET'])
 def get_reconciliation_status():
     """Return the result of the last startup reconciliation from /tmp status file."""
@@ -3478,6 +3502,8 @@ def get_reconciliation_status():
     try:
         with open(_recon_path) as _f:
             data = json.load(_f)
+        if data.get('unresolved'):
+            data['unresolved'] = _drop_stale_reconciliation_findings(data['unresolved'])
         return jsonify({'status': 'success', 'data': data})
     except json.JSONDecodeError:
         logger.exception("[Reconciliation] Failed to parse status file: %s", _recon_path)

web_interface/templates/v3/partials/overview.html

@@ -30,10 +30,33 @@
                 if (!d.unresolved || d.unresolved.length === 0) return;
                 var key = d.unresolved.map(function (i) { return i.plugin_id; }).sort().join(',');
                 if (sessionStorage.getItem(DISMISS_KEY) === key) return;
-                var ids = d.unresolved.map(function (i) { return i.plugin_id; }).join(', ');
-                document.getElementById('reconciliation-banner-text').textContent =
-                    'Stale plugin config entries found: ' + ids +
-                    '. Remove them from config.json or reinstall via the Plugin Store.';
+                // These are opposite problems and need opposite advice. Lumping
+                // them together told users to delete config.json entries for
+                // plugins that were installed and correctly configured.
+                function idsOfType(t) {
+                    return d.unresolved.filter(function (i) { return i.type === t; })
+                                       .map(function (i) { return i.plugin_id; });
+                }
+                var notConfigured = idsOfType('plugin_missing_in_config');
+                var notInstalled = idsOfType('plugin_missing_on_disk');
+                var other = d.unresolved.filter(function (i) {
+                    return i.type !== 'plugin_missing_in_config'
+                        && i.type !== 'plugin_missing_on_disk';
+                }).map(function (i) { return i.plugin_id; });
+
+                var parts = [];
+                if (notConfigured.length) {
+                    parts.push('Installed but missing from config: ' + notConfigured.join(', ')
+                        + '. Configure them in the Plugin Store - do not remove anything from config.json.');
+                }
+                if (notInstalled.length) {
+                    parts.push('In config but not installed: ' + notInstalled.join(', ')
+                        + '. Reinstall via the Plugin Store, or remove these entries from config.json.');
+                }
+                if (other.length) {
+                    parts.push('Needs attention: ' + other.join(', ') + '.');
+                }
+                document.getElementById('reconciliation-banner-text').textContent = parts.join(' ');
                 var banner = document.getElementById('reconciliation-banner');
                 banner.dataset.dismissKey = key;
                 banner.style.setProperty('display', 'flex', 'important');