refactor(plugins): one plugin-directory resolver (#623)

* refactor(plugins): one resolver for plugin id -> directory

Five places mapped a plugin id to its directory, each with its own rules
and each re-reading manifests per lookup: PluginManager discovery and
get_plugin_directory, PluginLoader.find_plugin_directory,
PluginStoreManager._find_plugin_path / list_installed_plugins, and
state_reconciliation.disk_plugin_ids. They disagreed on backup dirs,
on whether the manifest id or the directory name is the id, on duplicate
ids and on path safety.

src/plugin_system/plugin_dirs.py now holds the rules once:
PluginDirectoryIndex scans one directory and reads each manifest once;
resolve_plugin_dir() searches directories in order. What legitimately
differs per caller is an explicit argument: search dirs (discovery and
the loader: configured dir only; the store: configured then sibling
plugins/), ledmatrix- prefix (not for the store), case folding (loader
only), manifest pass (not for get_plugin_directory, whose discovery map
already holds it).

Behaviour changes, all for layouts installs do not produce:
- a directory whose manifest declares the id beats one merely named for
  it (discovery already worked this way; the loader and store now agree)
- the store searches the configured dir completely before plugins/
- backup and hidden dirs are skipped everywhere (the loader's case and
  manifest scans and list_installed_plugins used to return them)
- duplicate ids resolve deterministically (exact name, then
  ledmatrix-<id>, then by name) with a one-time warning; discovery no
  longer lists the id twice
- disk_plugin_ids / list_installed_plugins report manifest ids, falling
  back to the directory name; auto-update looks the directory up
- ids that are not one plain path segment resolve to nothing in every
  caller (the loader used to truncate them, the store to join them)

The .standalone-backup- marker is one constant, BACKUP_MARKER, used by
store_manager's rename-aside names and every lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): one plugin-directory resolver

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

src/plugin_system/plugin_dirs.py

@@ -0,0 +1,345 @@
+"""
+One answer to "which directory holds plugin X?".
+
+Five places used to answer it, each with its own rules and each re-parsing
+every manifest per lookup: ``PluginManager`` discovery and
+``get_plugin_directory``, ``PluginLoader.find_plugin_directory``,
+``PluginStoreManager._find_plugin_path`` / ``list_installed_plugins`` and
+``state_reconciliation.disk_plugin_ids``. The rules now live here once; what
+still legitimately differs between callers (which directories to search,
+whether a ``ledmatrix-`` prefix or a case difference counts as a match) is a
+keyword argument at the call site, so a difference is always a visible choice
+rather than an accident of which copy you read.
+
+The rules
+---------
+* A directory is a *candidate* when it is a directory (a symlink to one counts:
+  dev plugins are symlinked in) and its name is neither hidden (leading ``.``)
+  nor carries :data:`BACKUP_MARKER`. store_manager renames a plugin aside with
+  that marker during install/rollback; the aside still holds a manifest, so
+  treating it as a plugin would resurrect a ghost.
+* A plugin's id is its manifest ``id``. The directory name is only a fallback,
+  for callers that must still see a plugin whose manifest is missing an id.
+* Resolving an id within one directory: a directory whose manifest declares
+  the id wins; among several, the one named exactly for the id, then
+  ``ledmatrix-<id>``, then by name. Only when no manifest claims the id do
+  directory names count: ``<id>``, then ``ledmatrix-<id>`` (``prefix=True``),
+  then either of those ignoring case (``case_insensitive=True``). The name
+  fallback still returns a directory whose manifest is unreadable -- that is
+  how a broken plugin gets uninstalled or reinstalled. ``by_manifest=False``
+  (``PluginManager.get_plugin_directory``, whose discovery map already holds
+  the manifest answer) skips straight to the names.
+* Several directories are searched one at a time, in the order given: the
+  first directory that resolves the id at all wins, by manifest or by name.
+* The id must be one plain path segment (``safe_path_component``); anything
+  else resolves to nothing rather than being joined or truncated.
+* Returned paths are ``search_dir / name`` and are not resolved, so a
+  symlinked dev plugin keeps the path that lies inside the search directory.
+
+Manifests are read at most once per :class:`PluginDirectoryIndex`; one index is
+one scan.
+"""
+
+from __future__ import annotations
+
+import json
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any, Dict, Iterable, List, Optional, Set, Union
+
+from src.common.path_safety import safe_path_component
+
+__all__ = [
+    'BACKUP_MARKER',
+    'PLUGIN_DIR_PREFIX',
+    'ManifestStatus',
+    'PluginDirEntry',
+    'PluginDirectoryIndex',
+    'is_ignored_dir_name',
+    'resolve_plugin_dir',
+    'store_search_dirs',
+]
+
+#: Substring store_manager embeds in a plugin directory it has set aside
+#: (``<id>.standalone-backup-preinstall`` / ``-migrating``). Existing debris on
+#: devices carries exactly this text, so it must never change.
+BACKUP_MARKER = '.standalone-backup-'
+
+#: Legacy repository naming (``ledmatrix-<id>``); some installs still use it
+#: as the directory name.
+PLUGIN_DIR_PREFIX = 'ledmatrix-'
+
+PathLike = Union[str, Path]
+
+
+class ManifestStatus:
+    """What reading ``manifest.json`` in a candidate directory produced."""
+    OK = 'ok'                    # a JSON object with a non-empty "id"
+    MISSING = 'missing'          # no manifest.json
+    UNREADABLE = 'unreadable'    # I/O error or invalid JSON
+    NOT_OBJECT = 'not_object'    # valid JSON, but not an object
+    NO_ID = 'no_id'              # an object without a usable "id"
+
+
+def is_ignored_dir_name(name: str) -> bool:
+    """True for names that are never a plugin: hidden, or set aside mid-install."""
+    return name.startswith('.') or BACKUP_MARKER in name
+
+
+@dataclass
+class PluginDirEntry:
+    """One candidate directory and its manifest, read once."""
+    path: Path
+    status: str
+    manifest: Optional[Any] = None
+    error: Optional[BaseException] = None
+
+    @property
+    def name(self) -> str:
+        return self.path.name
+
+    @property
+    def manifest_id(self) -> Optional[str]:
+        """The manifest's ``id`` when the manifest is usable, else None."""
+        if self.status != ManifestStatus.OK:
+            return None
+        return self.manifest['id']
+
+    @property
+    def manifest_parses(self) -> bool:
+        """The manifest exists and is valid JSON (of any shape)."""
+        return self.status in (ManifestStatus.OK, ManifestStatus.NOT_OBJECT,
+                               ManifestStatus.NO_ID)
+
+    @property
+    def installed_id(self) -> str:
+        """The manifest id, falling back to the directory name."""
+        return self.manifest_id or self.name
+
+
+def _read_entry(path: Path) -> PluginDirEntry:
+    manifest_path = path / 'manifest.json'
+    if not manifest_path.is_file():
+        return PluginDirEntry(path, ManifestStatus.MISSING)
+    try:
+        with open(manifest_path, 'r', encoding='utf-8') as handle:
+            manifest = json.load(handle)
+    except (OSError, ValueError) as exc:  # ValueError covers JSON + decode errors
+        return PluginDirEntry(path, ManifestStatus.UNREADABLE, error=exc)
+    if not isinstance(manifest, dict):
+        return PluginDirEntry(path, ManifestStatus.NOT_OBJECT, manifest)
+    plugin_id = manifest.get('id')
+    if not plugin_id or not isinstance(plugin_id, str):
+        return PluginDirEntry(path, ManifestStatus.NO_ID, manifest)
+    return PluginDirEntry(path, ManifestStatus.OK, manifest)
+
+
+def _preference(plugin_id: str, name: str) -> tuple:
+    """Sort key among directories that all claim ``plugin_id``."""
+    if name == plugin_id:
+        rank = 0
+    elif name == PLUGIN_DIR_PREFIX + plugin_id:
+        rank = 1
+    else:
+        rank = 2
+    return (rank, name)
+
+
+@dataclass
+class PluginDirectoryIndex:
+    """Every candidate directory directly under ``root``, manifests read once.
+
+    Build one with :meth:`scan`. It is a snapshot: a directory added or
+    removed afterwards is not seen until the next scan.
+    """
+    root: Path
+    entries: List[PluginDirEntry] = field(default_factory=list)
+    #: Set when ``root`` exists but could not be listed.
+    error: Optional[BaseException] = None
+    _plugins: Optional[Dict[str, PluginDirEntry]] = field(
+        default=None, init=False, repr=False, compare=False)
+
+    @classmethod
+    def scan(cls, root: PathLike) -> 'PluginDirectoryIndex':
+        root = Path(root)
+        index = cls(root)
+        try:
+            children = sorted(root.iterdir(), key=lambda p: p.name)
+        except FileNotFoundError:
+            return index
+        except OSError as exc:
+            index.error = exc
+            return index
+        for child in children:
+            if is_ignored_dir_name(child.name):
+                continue
+            try:
+                if not child.is_dir():
+                    continue
+            except OSError:
+                continue
+            index.entries.append(_read_entry(child))
+        return index
+
+    # -- listing ----------------------------------------------------------
+
+    def plugins(self) -> Dict[str, PluginDirEntry]:
+        """Manifest id -> entry, one entry per id.
+
+        When several directories declare the same id, the one named for it
+        wins, then ``ledmatrix-<id>``, then the first by name; see
+        :meth:`duplicates` for the losers.
+        """
+        if self._plugins is not None:
+            return self._plugins
+        chosen: Dict[str, PluginDirEntry] = {}
+        for entry in self.entries:
+            plugin_id = entry.manifest_id
+            if plugin_id is None:
+                continue
+            current = chosen.get(plugin_id)
+            if current is None or (_preference(plugin_id, entry.name)
+                                   < _preference(plugin_id, current.name)):
+                chosen[plugin_id] = entry
+        self._plugins = chosen
+        return chosen
+
+    def duplicates(self) -> Dict[str, List[PluginDirEntry]]:
+        """Ids declared by more than one directory -> every such entry."""
+        seen: Dict[str, List[PluginDirEntry]] = {}
+        for entry in self.entries:
+            if entry.manifest_id is not None:
+                seen.setdefault(entry.manifest_id, []).append(entry)
+        return {k: v for k, v in seen.items() if len(v) > 1}
+
+    def installed_ids(self, *, require_parseable_manifest: bool) -> Set[str]:
+        """Ids of everything that counts as installed.
+
+        A directory counts when it has a manifest.json -- which must also be
+        valid JSON when ``require_parseable_manifest``. Its id is the manifest
+        id, or the directory name when the manifest does not carry one.
+        """
+        ids: Set[str] = set()
+        for entry in self.entries:
+            if entry.status == ManifestStatus.MISSING:
+                continue
+            if require_parseable_manifest and not entry.manifest_parses:
+                continue
+            ids.add(entry.installed_id)
+        return ids
+
+    def entry_for_installed_id(self, plugin_id: str) -> Optional[PluginDirEntry]:
+        """The entry :meth:`installed_ids` reported as ``plugin_id``."""
+        entry = self.plugins().get(plugin_id)
+        if entry is not None:
+            return entry
+        for entry in self.entries:
+            if entry.manifest_id is None and entry.name == plugin_id:
+                return entry
+        return None
+
+    # -- lookup -----------------------------------------------------------
+
+    def find(self, plugin_id: str, *, prefix: bool, case_insensitive: bool,
+             by_manifest: bool = True) -> Optional[Path]:
+        """Resolve ``plugin_id`` within this directory (rules in the module doc)."""
+        plugin_id = _lookup_id(plugin_id)
+        if plugin_id is None:
+            return None
+
+        if by_manifest:
+            entry = self.plugins().get(plugin_id)
+            if entry is not None:
+                return entry.path
+
+        names = _candidate_names(plugin_id, prefix)
+        by_name = {e.name: e for e in self.entries}
+        for name in names:
+            if name in by_name:
+                return by_name[name].path
+        if case_insensitive:
+            for low in (n.lower() for n in names):
+                for entry in self.entries:
+                    if entry.name.lower() == low:
+                        return entry.path
+        return None
+
+
+def _lookup_id(plugin_id: Any) -> Optional[str]:
+    """``plugin_id`` if it can name a plugin directory at all, else None."""
+    plugin_id = safe_path_component(plugin_id)
+    if plugin_id is None or is_ignored_dir_name(plugin_id):
+        return None
+    return plugin_id
+
+
+def _candidate_names(plugin_id: str, prefix: bool) -> List[str]:
+    names = [plugin_id]
+    if prefix:
+        names.append(PLUGIN_DIR_PREFIX + plugin_id)
+    return names
+
+
+def _is_dir(path: Path) -> bool:
+    try:
+        return path.is_dir()
+    except OSError:
+        return False
+
+
+def resolve_plugin_dir(plugin_id: Any, search_dirs: Iterable[PathLike], *,
+                       prefix: bool, case_insensitive: bool = False,
+                       by_manifest: bool = True) -> Optional[Path]:
+    """The directory holding ``plugin_id``, searching ``search_dirs`` in order.
+
+    Each search directory is scanned once and each manifest in it read once.
+    ``by_manifest=False`` skips the manifest pass and matches directory names
+    only, which reads no manifests at all.
+
+    Names are compared against the directory listing, never by probing
+    ``search_dir / name``: on a case-insensitive filesystem that probe says
+    ``Demo`` exists when the directory is ``demo``, which made the answer
+    depend on the platform.
+    """
+    plugin_id = _lookup_id(plugin_id)
+    if plugin_id is None:
+        return None
+    for search_dir in search_dirs:
+        search_dir = Path(search_dir)
+        if by_manifest or case_insensitive:
+            found = PluginDirectoryIndex.scan(search_dir).find(
+                plugin_id, prefix=prefix, case_insensitive=case_insensitive,
+                by_manifest=by_manifest)
+        else:
+            found = _find_by_name(search_dir, _candidate_names(plugin_id, prefix))
+        if found is not None:
+            return found
+    return None
+
+
+def _find_by_name(search_dir: Path, names: List[str]) -> Optional[Path]:
+    try:
+        present = {child.name for child in search_dir.iterdir()}
+    except OSError:
+        return None
+    for name in names:
+        if name in present and _is_dir(search_dir / name):
+            return search_dir / name
+    return None
+
+
+def store_search_dirs(plugins_dir: PathLike) -> List[Path]:
+    """Directories the plugin store searches: the configured one, then a
+    sibling ``plugins/`` (the legacy/dev location) when that is a different
+    directory. Discovery deliberately does NOT use this -- it scans only the
+    configured directory (see CLAUDE.md, test_discovery_path_contract.py)."""
+    plugins_dir = Path(plugins_dir)
+    dirs = [plugins_dir]
+    try:
+        base = plugins_dir if plugins_dir.is_absolute() else plugins_dir.resolve()
+        sibling = base.parent / 'plugins'
+        if sibling != base:
+            dirs.append(sibling)
+    except (OSError, ValueError):
+        pass
+    return dirs

src/plugin_system/plugin_loader.py

@@ -8,7 +8,6 @@ Extracted from PluginManager to improve separation of concerns.
 import importlib
 import importlib.metadata
 import importlib.util
-import json
 import os
 import sys
 import subprocess
@@ -21,6 +20,7 @@ from packaging.requirements import InvalidRequirement, Requirement
 
 from src.exceptions import PluginError
 from src.logging_config import get_logger
+from src.plugin_system.plugin_dirs import resolve_plugin_dir
 
 
 def requirements_has_real_deps(requirements_file: str) -> bool:
@@ -214,85 +214,36 @@ class PluginLoader:
     ) -> Optional[Path]:
         """
         Find the plugin directory for a given plugin ID.
-        
-        Tries multiple strategies:
-        1. Use plugin_directories mapping if available
-        2. Direct path matching
-        3. Case-insensitive directory matching
-        4. Manifest-based search
-        
+
+        1. The discovery mapping, when it has the id and the path exists.
+        2. ``plugins_dir`` only, by the shared rules in
+           ``src/plugin_system/plugin_dirs.py``: a directory whose manifest
+           declares the id wins; otherwise ``<id>`` or ``ledmatrix-<id>``,
+           matched case-insensitively. Backup and hidden directories are
+           never matched.
+
         Args:
             plugin_id: Plugin identifier
             plugins_dir: Base plugins directory
             plugin_directories: Optional mapping of plugin_id to directory
-            
-        Returns:
-            Path to plugin directory or None if not found
-        """
-        # Sanitize plugin_id — os.path.basename is a CodeQL-recognized path sanitizer
-        plugin_id = os.path.basename(plugin_id or '')
-        if not plugin_id:
-            return None
 
+        Returns:
+            Path to plugin directory or None if not found. An id that is not
+            one plain path segment finds nothing.
+        """
         # Strategy 1: Use mapping from discovery
         if plugin_directories and plugin_id in plugin_directories:
             plugin_dir = plugin_directories[plugin_id]
             if plugin_dir.exists():
                 self.logger.debug("Using plugin directory from discovery mapping: %s", plugin_dir)
                 return plugin_dir
-        
-        # Strategy 2: Direct paths — resolve and validate they stay within plugins_dir
-        plugins_dir_resolved = plugins_dir.resolve()
-        for _candidate_name in (plugin_id, f"ledmatrix-{plugin_id}"):
-            _candidate = (plugins_dir_resolved / _candidate_name).resolve()
-            try:
-                _candidate.relative_to(plugins_dir_resolved)
-            except ValueError:
-                continue
-            if _candidate.exists():
-                return _candidate
-        
-        # Strategy 3: Case-insensitive search
-        normalized_id = plugin_id.lower()
-        for item in plugins_dir.iterdir():
-            if not item.is_dir():
-                continue
-            
-            item_name = item.name
-            if item_name.lower() == normalized_id:
-                return item
-            
-            if item_name.lower() == f"ledmatrix-{plugin_id}".lower():
-                return item
-        
-        # Strategy 4: Manifest-based search
-        self.logger.debug("Directory name search failed for %s, searching by manifest...", plugin_id)
-        for item in plugins_dir.iterdir():
-            if not item.is_dir():
-                continue
-            
-            # Skip if already checked
-            if item.name.lower() == normalized_id or item.name.lower() == f"ledmatrix-{plugin_id}".lower():
-                continue
-            
-            manifest_path = item / "manifest.json"
-            if manifest_path.exists():
-                try:
-                    with open(manifest_path, 'r', encoding='utf-8') as f:
-                        item_manifest = json.load(f)
-                        item_manifest_id = item_manifest.get('id')
-                        if item_manifest_id == plugin_id:
-                            self.logger.info(
-                                "Found plugin %s in directory %s (manifest ID matches)",
-                                plugin_id,
-                                item.name
-                            )
-                            return item
-                except (json.JSONDecodeError, Exception) as e:
-                    self.logger.debug("Skipping %s due to manifest error: %s", item.name, e)
-                    continue
 
-        return None
+        plugin_dir = resolve_plugin_dir(
+            plugin_id, [plugins_dir], prefix=True, case_insensitive=True)
+        if plugin_dir is not None and plugin_dir.name != plugin_id:
+            self.logger.debug("Found plugin %s in directory %s",
+                              plugin_id, plugin_dir.name)
+        return plugin_dir
 
     def install_dependencies(
         self,

src/plugin_system/plugin_manager.py

@@ -25,7 +25,9 @@ from src.plugin_system.plugin_state import PluginStateManager, PluginState
 from src.plugin_system.schema_manager import (
     CORE_VEGAS_TUNING_KEYS, SchemaManager, normalize_legacy_booleans,
 )
-from src.common.path_safety import safe_path_component
+from src.plugin_system.plugin_dirs import (
+    ManifestStatus, PluginDirectoryIndex, resolve_plugin_dir,
+)
 from src.deprecation import deprecated
 from src.common.permission_utils import (
     ensure_directory_permissions,
@@ -175,89 +177,88 @@ class PluginManager:
             self.logger.error("Could not create plugins directory %s: %s", self.plugins_dir, e, exc_info=True)
             raise PluginError(f"Could not create plugins directory: {self.plugins_dir}", context={'error': str(e)}) from e
 
+    def _report_skip_once(self, key: str, message: str, *args: Any) -> None:
+        """Warn about a skipped directory once per process, not per scan.
+
+        Discovery runs on every web UI page load and every config reconcile,
+        so warning unconditionally would put a line in the journal each time
+        someone opened a page -- the same log-volume problem this is meant to
+        help diagnose.
+        """
+        reported = self.__dict__.setdefault('_skip_reported', set())
+        if key in reported:
+            return
+        reported.add(key)
+        self.logger.warning(message, *args)
+
     def _scan_directory_for_plugins(self, directory: Path) -> List[str]:
         """
         Scan a directory for plugins.
 
+        Which directories count and how an id maps to one is decided by
+        :class:`PluginDirectoryIndex` (``src/plugin_system/plugin_dirs.py``),
+        shared with the loader, the store and reconciliation. Only
+        ``directory`` is scanned: discovery has no fallback to ``plugins/``.
+        Directories set aside mid-install (``BACKUP_MARKER`` in the name) are
+        skipped so they don't overwrite live entries.
+
         Args:
             directory: Directory to scan
 
         Returns:
             List of plugin IDs found
         """
-        plugin_ids = []
-
         if not directory.exists():
-            return plugin_ids
+            return []
 
         # Build new state locally before acquiring lock
-        new_manifests: Dict[str, Dict[str, Any]] = {}
-        new_directories: Dict[str, Path] = {}
-
-        try:
-            for item in directory.iterdir():
-                if not item.is_dir():
-                    continue
-                # Skip backup directories so they don't overwrite live entries
-                if '.standalone-backup-' in item.name:
-                    continue
-
-                manifest_path = item / "manifest.json"
-                if not manifest_path.exists():
-                    # Once per directory per process. Discovery runs on every
-                    # web UI page load and every config reconcile, so warning
-                    # unconditionally would put a line in the journal each
-                    # time someone opened a page -- the same log-volume
-                    # problem this is meant to help diagnose.
-                    # A directory here that carries no manifest is not a
-                    # plugin. Said once, because the alternative is a plugin
-                    # that is enabled in config, enabled in plugin state,
-                    # present on disk, and simply absent from the running
-                    # process with nothing anywhere to say why. Working that
-                    # out afterwards means reading cache-file mtimes.
-                    if item.name not in self._skip_reported:
-                        self._skip_reported.add(item.name)
-                        self.logger.warning(
-                            "Skipping %s: no manifest.json, so it cannot be "
-                            "loaded as a plugin", item.name)
-                    continue
-                try:
-                    with open(manifest_path, 'r', encoding='utf-8') as f:
-                        manifest = json.load(f)
-                except (json.JSONDecodeError, PermissionError, OSError) as e:
-                    self.logger.warning("Error reading manifest from %s: %s", manifest_path, e, exc_info=True)
-                    continue
+        index = PluginDirectoryIndex.scan(directory)
+        if index.error is not None:
+            self.logger.error("Error scanning directory %s: %s", directory,
+                              index.error, exc_info=index.error)
 
+        for entry in index.entries:
+            if entry.status == ManifestStatus.MISSING:
+                # A directory here that carries no manifest is not a plugin.
+                # Said once, because the alternative is a plugin that is
+                # enabled in config, enabled in plugin state, present on disk,
+                # and simply absent from the running process with nothing
+                # anywhere to say why. Working that out afterwards means
+                # reading cache-file mtimes.
+                self._report_skip_once(
+                    entry.name, "Skipping %s: no manifest.json, so it cannot be "
+                    "loaded as a plugin", entry.name)
+            elif entry.status == ManifestStatus.UNREADABLE:
+                self.logger.warning("Error reading manifest from %s: %s",
+                                    entry.path / "manifest.json", entry.error,
+                                    exc_info=entry.error)
+            elif entry.status == ManifestStatus.NOT_OBJECT:
                 # json.load accepts any JSON value, so a manifest holding
-                # null, [] or "text" parses and then raises AttributeError on
-                # .get(). Nothing here catches that -- the outer handler takes
-                # OSError/PermissionError only -- so a single malformed
-                # manifest aborted the whole scan and every other plugin on
+                # null, [] or "text" parses. It once raised AttributeError on
+                # .get() and aborted the whole scan, so every other plugin on
                 # disk, however healthy, silently failed to register.
-                if not isinstance(manifest, dict):
-                    if item.name not in self._skip_reported:
-                        self._skip_reported.add(item.name)
-                        self.logger.warning(
-                            "Skipping %s: its manifest.json is %s, not a JSON "
-                            "object", item.name, type(manifest).__name__)
-                    continue
+                self._report_skip_once(
+                    entry.name, "Skipping %s: its manifest.json is %s, not a "
+                    "JSON object", entry.name, type(entry.manifest).__name__)
+            elif entry.status == ManifestStatus.NO_ID:
+                # Parsed but unusable. This was the quietest path of all: the
+                # manifest is read successfully and then dropped.
+                self._report_skip_once(
+                    entry.name, "Skipping %s: its manifest.json has no \"id\", "
+                    "so there is nothing to register it under", entry.name)
 
-                plugin_id = manifest.get('id')
-                if not plugin_id:
-                    # Parsed but unusable. This was the quietest path of all:
-                    # the manifest is read successfully and then dropped.
-                    if item.name not in self._skip_reported:
-                        self._skip_reported.add(item.name)
-                        self.logger.warning(
-                            "Skipping %s: its manifest.json has no \"id\", so "
-                            "there is nothing to register it under", item.name)
-                    continue
+        plugins = index.plugins()
+        for plugin_id, entries in index.duplicates().items():
+            self._report_skip_once(
+                "duplicate:" + plugin_id,
+                "Plugin id %r is declared by %d directories (%s); using %s",
+                plugin_id, len(entries), ", ".join(e.name for e in entries),
+                plugins[plugin_id].name)
 
-                plugin_ids.append(plugin_id)
-                new_manifests[plugin_id] = manifest
-                new_directories[plugin_id] = item
-        except (OSError, PermissionError) as e:
-            self.logger.error("Error scanning directory %s: %s", directory, e, exc_info=True)
+        new_manifests: Dict[str, Dict[str, Any]] = {
+            plugin_id: entry.manifest for plugin_id, entry in plugins.items()}
+        new_directories: Dict[str, Path] = {
+            plugin_id: entry.path for plugin_id, entry in plugins.items()}
 
         # Replace shared state under lock so uninstalled plugins don't linger
         with self._discovery_lock:
@@ -266,8 +267,8 @@ class PluginManager:
             self.plugin_directories.clear()
             self.plugin_directories.update(new_directories)
 
-        return plugin_ids
-    
+        return list(plugins)
+
     def discover_plugins(self) -> List[str]:
         """
         Discover all plugins in the plugins directory.
@@ -772,24 +773,20 @@ class PluginManager:
         not one plain path segment (``..``, ``a/b``, an absolute path) is
         refused instead of being joined onto ``plugins_dir``. The join is not
         resolved further: dev plugins are symlinks into ``plugins_dir``.
+
+        The discovery map is authoritative. For an id discovery has not seen,
+        only directory names are tried -- ``<id>`` then ``ledmatrix-<id>``,
+        in ``plugins_dir`` only -- so a miss on a web request never reads
+        every manifest on disk. Rules: ``src/plugin_system/plugin_dirs.py``.
         """
         with self._discovery_lock:
             if plugin_id in self.plugin_directories:
                 return str(self.plugin_directories[plugin_id])
 
-        plugin_id = safe_path_component(plugin_id)
-        if plugin_id is None:
-            return None
-
-        plugin_dir = self.plugins_dir / plugin_id
-        if plugin_dir.exists():
-            return str(plugin_dir)
-        
-        plugin_dir = self.plugins_dir / f"ledmatrix-{plugin_id}"
-        if plugin_dir.exists():
-            return str(plugin_dir)
-        
-        return None
+        plugin_dir = resolve_plugin_dir(
+            plugin_id, [self.plugins_dir], prefix=True, case_insensitive=False,
+            by_manifest=False)
+        return str(plugin_dir) if plugin_dir is not None else None
     
     def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
         """

src/plugin_system/state_reconciliation.py

@@ -15,6 +15,7 @@ from enum import Enum
 from pathlib import Path
 
 from src.core_config_keys import CORE_CONFIG_KEYS
+from src.plugin_system.plugin_dirs import PluginDirectoryIndex
 from src.plugin_system.state_manager import PluginStateManager
 from src.logging_config import get_logger
 
@@ -102,31 +103,25 @@ def config_plugin_ids(config: Dict[str, Any], ignored_keys: Set[str]) -> Set[str
 def disk_plugin_ids(plugins_dir) -> Set[str]:
     """Plugin ids actually installed on disk.
 
-    A directory counts only when it is not a standalone backup and its
-    manifest.json parses. A corrupt manifest must not read as installed, or a
-    live "in config but not on disk" finding gets cleared on the strength of an
-    unreadable file.
+    A directory counts only when it is not a standalone backup (or hidden)
+    and its manifest.json parses. A corrupt manifest must not read as
+    installed, or a live "in config but not on disk" finding gets cleared on
+    the strength of an unreadable file.
+
+    The id is the manifest's ``id`` -- what discovery registers and what the
+    config is keyed by -- and the directory name only when the manifest has
+    none. Directory names alone made a plugin living in ``ledmatrix-stocks/``
+    with id ``stocks`` read as both "stocks in config but not on disk" and
+    "ledmatrix-stocks on disk but not in config".
     """
-    ids: Set[str] = set()
-    root = Path(plugins_dir)
     try:
-        if not root.exists():
-            return ids
-        for entry in root.iterdir():
-            if not entry.is_dir() or '.standalone-backup-' in entry.name:
-                continue
-            manifest = entry / "manifest.json"
-            if not manifest.exists():
-                continue
-            try:
-                with open(manifest, 'r') as f:
-                    json.load(f)
-            except (OSError, ValueError):
-                continue
-            ids.add(entry.name)
+        return _disk_index(plugins_dir).installed_ids(require_parseable_manifest=True)
     except OSError:
-        return ids
-    return ids
+        return set()
+
+
+def _disk_index(plugins_dir) -> PluginDirectoryIndex:
+    return PluginDirectoryIndex.scan(Path(plugins_dir))
 
 
 def still_unresolved(entries: List[Dict[str, Any]],
@@ -326,16 +321,15 @@ class StateReconciliation:
         """Get plugin state from disk (installed plugins)."""
         state = {}
         try:
-            # Membership comes from the shared extractor so the web interface
-            # re-checks stored findings against this same definition; the
-            # manifest is then re-read here only for version/name.
-            for plugin_id in disk_plugin_ids(self.plugins_dir):
-                manifest_path = self.plugins_dir / plugin_id / "manifest.json"
-                try:
-                    with open(manifest_path, 'r') as f:
-                        manifest = json.load(f)
-                except (OSError, ValueError):  # nosec B112 - raced or corrupt; skip
-                    continue
+            # Membership uses the same index and rule as disk_plugin_ids, so
+            # the web interface re-checks stored findings against this same
+            # definition; each manifest is read once, by the scan.
+            index = _disk_index(self.plugins_dir)
+            for plugin_id in index.installed_ids(require_parseable_manifest=True):
+                entry = index.entry_for_installed_id(plugin_id)
+                manifest = entry.manifest if entry is not None else None
+                if not isinstance(manifest, dict):
+                    manifest = {}
                 state[plugin_id] = {
                     'exists_on_disk': True,
                     'version': manifest.get('version'),

src/plugin_system/store_manager.py

@@ -28,6 +28,9 @@ from src.common.permission_utils import sudo_remove_directory, install_requireme
 from src.plugin_system.plugin_loader import (
     requirements_has_real_deps, requirements_are_satisfied, find_trusted_subdir
 )
+from src.plugin_system.plugin_dirs import (
+    BACKUP_MARKER, PluginDirectoryIndex, resolve_plugin_dir, store_search_dirs,
+)
 
 try:
     from jsonschema import Draft7Validator, ValidationError
@@ -1233,9 +1236,9 @@ class PluginStoreManager:
         Pass-through when nothing is installed, and when called from
         `_reinstall_with_rollback`, which has already moved the old copy aside.
 
-        The aside name embeds '.standalone-backup-' so plugin discovery
-        (`plugin_manager._scan_directory_for_plugins`) skips it even though it
-        still holds a manifest.json.
+        The aside name embeds BACKUP_MARKER ('.standalone-backup-') so every
+        plugin directory lookup (src/plugin_system/plugin_dirs.py) skips it
+        even though it still holds a manifest.json.
 
         Held under the per-plugin reinstall lock for the same reason
         `_reinstall_with_rollback` is: the web UI runs Flask with
@@ -1250,7 +1253,7 @@ class PluginStoreManager:
                 return self._install_plugin_impl(plugin_id, branch)
 
             backup_path = plugin_path.with_name(
-                f"{plugin_path.name}.standalone-backup-preinstall")
+                f"{plugin_path.name}{BACKUP_MARKER}preinstall")
             if backup_path.exists() and not self._safe_remove_directory(backup_path):
                 # Can't stage a safety net. Better to attempt the install than
                 # to refuse outright, which is what callers got before this
@@ -2393,95 +2396,43 @@ class PluginStoreManager:
     def _find_plugin_path(self, plugin_id: str) -> Optional[Path]:
         """
         Find the plugin path by checking the configured directory and standard plugins directory.
-        
+
+        Searches the configured directory, then a sibling ``plugins/`` (the
+        case where plugins sit in plugins/ but config says plugin-repos/) --
+        a store-only fallback; discovery scans the configured directory only.
+        Each directory is searched completely before the next, by the shared
+        rules in ``src/plugin_system/plugin_dirs.py``: a directory whose
+        manifest declares the id wins, then a directory named exactly for it.
+
+        The manifest match matters because a directory name can differ from
+        the id its manifest declares (a hand-made or legacy layout such as
+        `ledmatrix-stocks/` holding id `stocks`); a lookup by directory name
+        alone reported such a plugin as not installed, so update_plugin()
+        silently did nothing.
+
+        No ``ledmatrix-`` prefix and no case folding here, unlike the loader:
+        a store operation may delete what this returns, so it only accepts a
+        directory that names the id exactly or declares it. Note that this
+        leaves registry ids like `stocks` unresolved when the installed
+        plugin is `ledmatrix-stocks/` declaring `ledmatrix-stocks` (the
+        monorepo's leaderboard, music, stocks and weather); passing
+        ``prefix=True`` would resolve them, but update_plugin()'s reinstall
+        path has not been checked against that yet.
+
         Args:
             plugin_id: Plugin identifier
-            
+
         Returns:
             Path to plugin directory if found, None otherwise
         """
-        # First check the configured plugins directory
-        plugin_path = self.plugins_dir / plugin_id
-        if plugin_path.exists():
-            return plugin_path
-        
-        # Also check the standard 'plugins/' directory if it's different
-        # This handles the case where plugins are in plugins/ but config says plugin-repos/
-        try:
-            if self.plugins_dir.is_absolute():
-                project_root = self.plugins_dir.parent
-            else:
-                project_root = self.plugins_dir.resolve().parent
-            
-            standard_plugins_dir = project_root / 'plugins'
-            if standard_plugins_dir.exists() and standard_plugins_dir != self.plugins_dir:
-                plugin_path = standard_plugins_dir / plugin_id
-                if plugin_path.exists():
-                    return plugin_path
-        except (OSError, ValueError):
-            pass
-
-        # Last resort: the directory name may differ from the id being looked
-        # up. install_plugin() deliberately renames a plugin's directory to the
-        # MANIFEST id when it differs from the REGISTRY id (see the rename near
-        # "doesn't match registry ID" above), so `stocks` in the registry lands
-        # in `ledmatrix-stocks/`. Every lookup above is by directory name, so
-        # update_plugin("stocks") found nothing and reported the plugin as not
-        # installed -- silently, and for good: the user sees no error and stays
-        # on a stale version. Four installed plugins hit this in practice
-        # (leaderboard, music, stocks, weather).
-        #
-        # Deliberately last so the two lookups above keep their exact meaning;
-        # this only runs when a direct hit already failed. See
-        # test_discovery_path_contract.py, which pins that ordering.
-        for search_dir in self._candidate_plugin_dirs():
-            match = self._find_by_manifest_id(search_dir, plugin_id)
-            if match is not None:
-                self.logger.debug(
-                    "Resolved plugin '%s' to %s via its manifest id "
-                    "(directory name differs from the id)", plugin_id, match)
-                return match
-
-        return None
+        return resolve_plugin_dir(
+            plugin_id, self._candidate_plugin_dirs(), prefix=False,
+            case_insensitive=False)
 
     def _candidate_plugin_dirs(self) -> List[Path]:
         """Directories that may hold installed plugins, configured one first."""
-        dirs = [self.plugins_dir]
-        try:
-            base = self.plugins_dir if self.plugins_dir.is_absolute()                 else self.plugins_dir.resolve()
-            sibling = base.parent / 'plugins'
-            if sibling != self.plugins_dir:
-                dirs.append(sibling)
-        except (OSError, ValueError):
-            pass
-        return [d for d in dirs if d.exists()]
+        return [d for d in store_search_dirs(self.plugins_dir) if d.exists()]
 
-    @staticmethod
-    def _find_by_manifest_id(search_dir: Path, plugin_id: str) -> Optional[Path]:
-        """A subdirectory of `search_dir` whose manifest declares `plugin_id`.
-
-        Skips half-finished installs: store_manager renames a directory aside
-        with '.standalone-backup-' during install and rollback, and treating
-        one as installed would resurrect a ghost plugin.
-        """
-        try:
-            entries = sorted(search_dir.iterdir())
-        except (OSError, ValueError):
-            return None
-        for entry in entries:
-            if not entry.is_dir() or '.standalone-backup-' in entry.name:
-                continue
-            manifest = entry / 'manifest.json'
-            if not manifest.is_file():
-                continue
-            try:
-                with open(manifest, 'r', encoding='utf-8') as handle:
-                    if json.load(handle).get('id') == plugin_id:
-                        return entry
-            except (OSError, ValueError):
-                continue
-        return None
-    
     def uninstall_plugin(self, plugin_id: str) -> bool:
         """
         Uninstall a plugin by removing its directory.
@@ -2600,9 +2551,9 @@ class PluginStoreManager:
         field during the monorepo migration on a Pi with broken DNS — every
         old-remote plugin was deleted and none could be re-downloaded).
 
-        The aside name embeds '.standalone-backup-' so plugin discovery
-        (plugin_manager._scan_directory_for_plugins) ignores it even though
-        it still contains a manifest.json.
+        The aside name embeds BACKUP_MARKER ('.standalone-backup-') so every
+        plugin directory lookup (src/plugin_system/plugin_dirs.py) ignores it
+        even though it still contains a manifest.json.
 
         Held for the whole operation under a per-plugin_id lock: two
         overlapping requests for the same plugin (double-click, two
@@ -2612,7 +2563,7 @@ class PluginStoreManager:
         """
         with self._get_reinstall_lock(plugin_id):
             backup_path = plugin_path.with_name(
-                f"{plugin_path.name}.standalone-backup-migrating")
+                f"{plugin_path.name}{BACKUP_MARKER}migrating")
             # A stale aside from a previous crash would block the rename
             if backup_path.exists():
                 if not self._safe_remove_directory(backup_path):
@@ -3084,16 +3035,16 @@ class PluginStoreManager:
     def list_installed_plugins(self) -> List[str]:
         """
         Get list of installed plugin IDs.
-        
+
+        One entry per plugin directory in the configured directory that has a
+        manifest.json, named by the manifest's id (the directory name when
+        the manifest carries none, e.g. because it does not parse). Backup
+        and hidden directories are not plugins.
+
         Returns:
-            List of plugin IDs
+            List of plugin IDs, sorted
         """
         if not self.plugins_dir.exists():
             return []
-        
-        installed = []
-        for item in self.plugins_dir.iterdir():
-            if item.is_dir() and (item / "manifest.json").exists():
-                installed.append(item.name)
-        
-        return installed
+        index = PluginDirectoryIndex.scan(self.plugins_dir)
+        return sorted(index.installed_ids(require_parseable_manifest=False))