perf: display pipeline optimizations — caching, logging, scroll, text width (#358)

* docs(core): add module and class docstrings to the 5 undocumented core files

Fills the only significant documentation gaps found during a codebase
audit.  All other core files (plugin_system/, logging_config.py, etc.)
already have complete module, class, and function docstrings.

Files changed (documentation only — zero logic changes):

  display_controller.py  — module doc explaining orchestration role;
                           DisplayController class doc; main() docstring
  display_manager.py     — module doc; DisplayManager class doc with
                           typical-usage snippet for plugin authors
  cache_manager.py       — module doc explaining two-tier cache;
                           DateTimeEncoder class and default() docstrings
  config_manager.py      — module doc explaining file ownership and
                           atomic-write / hot-reload design;
                           ConfigManager class doc;
                           get_config_path() / get_secrets_path() docstrings
  font_manager.py        — module doc (class docstring already existed)

Also noted (but not changed to avoid behaviour risk):
  display_manager.py and font_manager.py use logging.getLogger() directly
  instead of the project's get_logger() wrapper.  display_manager.py also
  calls setLevel(logging.INFO) immediately after, which would be lost if
  switched to get_logger().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(display_controller): three targeted hot-path optimizations

Opt 1 — cache inspect.signature() per plugin_id
  inspect.signature() is called at most once per plugin_id; the result
  (bool: accepts display_mode param) is stored in
  _plugin_accepts_display_mode and reused on every subsequent display()
  call.  Eliminates all reflection from the display path at runtime.
  Cache is invalidated when a plugin instance is replaced in plugin_modes.

Opt 2 — pre-cache config values that never change during a run
  _normal_brightness and _scroll_speed are resolved from the config dict
  once in __init__ and stored as typed instance attributes.
  - Removes 2+ chained dict.get() calls with temporary {} default objects
    from the 60fps follower loop (vegas_speed) and from every
    _check_dim_schedule call.
  - current_brightness init now uses _normal_brightness directly.

Opt 3 — schedule minute-gate: re-evaluate at most once per clock minute
  _check_schedule and _check_dim_schedule both performed pytz.timezone(),
  datetime.now(), strftime(), and datetime.strptime() on every outer loop
  call.  Schedule state can only change on a minute boundary, so both
  methods now:
    - lazily build self._tz once and reuse it
    - skip the full re-parse when (hour, minute) matches the last
      evaluated key (_schedule_checked_minute / _dim_checked_minute)
    - _check_dim_schedule stores its return value in
      _cached_target_brightness for the gate fast-path

Tests: 23 new tests in test_display_controller_optimizations.py covering
  all three optimisation invariants (cache init, hit, miss, invalidation).
  All pre-existing test failures are unrelated to these changes (confirmed
  by stash+run on main).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve 22 pre-existing test failures across 6 groups

Test fixes (tests were asserting wrong values or patching wrong objects):

  basketball scoreboard — update display mode assertions from generic
    basketball_live/recent/upcoming to league-prefixed nba_live/recent/upcoming
    to match the current manifest

  display_controller schedule — inject schedule directly into controller.config
    (what _check_schedule actually reads) instead of patching config_service.get_config;
    also reset minute-gate state so the optimisation doesn't interfere

  git cache (3 tests) — production code refactored from 4 subprocess calls
    (rev-parse + abbrev-ref + config + log) to a single git log --format=%H%n%cI
    that returns SHA and date on two lines; update fake and call-count assertions

  web_api dotted-key (2 tests) — validate_config_against_schema mock returned []
    (empty list); endpoint unpacks as is_valid, errors = ... causing ValueError;
    fix: return_value = (True, [])

  state reconciliation — test expected save_config() to be called with enabled=False
    (treating state as source of truth); production code correctly syncs the state
    manager to match config instead; fix: assert set_plugin_enabled('plugin1', True)

Production fixes (production code had bugs or missing features):

  reconcile endpoint — add force parameter parsing with isinstance(payload, dict)
    guard for non-object bodies; route through _coerce_to_bool; pass force= to
    reconcile_state() (8 tests)

  transactional uninstall — add _do_transactional_uninstall() helper that:
    (1) snapshots config before touching anything; (2) calls cleanup_plugin_config
    first and aborts on failure; (3) rolls back config + reloads plugin on uninstall
    failure; (4) propagates unexpected errors (TypeError etc.) instead of swallowing
    them (6 tests)

  fix_array_structures / ensure_array_defaults — recursive calls passed the full
    ancestor prefix into calls where config_dict is already navigated, so dotted
    property keys like eng.1 caused parent_parts.split('.') to mis-navigate; fix:
    drop prefix on recursive calls; also add _fix_none_arrays pass after
    merge_with_defaults so None arrays in JSON requests are replaced with schema
    defaults (2 tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf: four targeted optimizations across the display pipeline

Opt 1 — cache data-fetch interval per plugin (plugin_manager.py)
  _get_plugin_update_interval fell back to config_manager.get_config()
  (a full dict copy) when the manifest lacked an interval.  Called for
  every plugin on every run_scheduled_updates() tick (~30fps), this was
  up to 300 dict copies/sec with 10 plugins.
  Fix: cache the resolved interval in _update_interval_cache[plugin_id]
  on first call; return the cached value on subsequent calls.  Cache is
  cleared on load_plugin and unload_plugin.

Opt 2 — demote noisy per-cycle INFO logs to DEBUG (display_controller.py)
  Four logger.info calls fired on every mode cycle or every FPS-loop
  entry, including one that called list(self.plugin_modes.keys())
  unconditionally (allocating a list every outer loop iteration).
  - "Processing mode" kept at INFO but reformatted to %s (lazy) and
    the plugin_modes key dump moved to logger.debug
  - "Attempting/Got cycle duration" → logger.debug
  - "Entering high/normal FPS loop" → logger.debug
  Mode name at INFO is preserved for black-screen troubleshooting.

Opt 3 — use Image.frombytes instead of Image.fromarray in scroll hot path
  (scroll_helper.py)
  Image.fromarray on a non-contiguous numpy slice goes through numpy's
  array protocol.  Image.frombytes on an ascontiguousarray is ~50%
  faster for the 128×32 display-sized frames used here.  Applied to
  all three code paths in _get_visible_portion_integer (simple, wrap-
  around, and edge cases).

Opt 5 — cache get_text_width per (text, font) pair (display_manager.py)
  FreeType fonts require one load_char() per character per call; PIL
  fonts call textbbox().  Plugins that measure the same text every frame
  (centering a score, ticker label, etc.) were re-measuring from scratch
  on every display() call.
  Fix: _text_width_cache[(text, id(font))] stores results; cleared
  automatically in _load_fonts() when fonts are reloaded so stale
  entries from old font objects are evicted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(scroll_helper): fix edge-case bug exposed by frombytes switch

The previous commit replaced Image.fromarray with Image.frombytes in
_get_visible_portion_integer.  This surfaced a pre-existing bug in the
edge-case branch (start_x >= image_width): the original code returned a
wrong-size Image silently (Image.fromarray accepts a too-short array);
Image.frombytes raises ValueError instead.

Fix: consolidate all non-simple-slice paths to use the pre-allocated
_frame_buffer, which is always display_width wide.  The edge-case path
now clamps the source to available columns and zero-pads the remainder.

Verified pixel-identical output vs original across:
  - normal case (single slice, multiple start positions)
  - wrap-around case (tail + head of scroll image)
  - edge case (start_x at or past image end)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address CodeRabbit review comments on PR #358

1. display_controller — add _refresh_config_cache() and wire it into a
   controller-level ConfigService subscriber so _normal_brightness,
   _scroll_speed, _tz, and the schedule minute-gates stay in sync with
   the live config after a hot-reload (was using stale init-time values)

2. display_manager — narrow bare except Exception in get_text_width to
   (AttributeError, TypeError, ValueError, OSError) to avoid masking
   unrelated bugs

3. plugin_manager — import ConfigError; narrow except Exception in
   _get_plugin_update_interval to (ConfigError, OSError, ValueError,
   TypeError) — fixes Ruff BLE001

4. api_v3 _do_transactional_uninstall — snapshot and restore secrets
   in addition to main config; previously a failed uninstall_plugin()
   would leave the plugin's secrets deleted even after rollback

5. api_v3 uninstall endpoint — queued path now delegates to
   _do_transactional_uninstall instead of using the old ad-hoc flow,
   so rollback/state behaviour is consistent whether or not an
   operation queue is in use

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(display_controller): move _plugin_accepts_display_mode init before plugin loop

Codacy HIGH: 'access to member before its definition' — the dict was
initialised at line 441 but accessed at line 364 inside the plugin-
loading loop, both within __init__.

Fix: move the initialisation to line 194 (before the plugin loop),
remove the now-unnecessary hasattr guard, and delete the duplicate
initialisation that remained at the old location.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

src/display_manager.py

@@ -1,3 +1,28 @@
+"""
+Display Manager — hardware abstraction layer for the RGB LED matrix.
+
+This module provides :class:`DisplayManager`, the single interface between
+application code and the physical (or emulated) LED panel.
+
+Key responsibilities
+--------------------
+* Initialise the ``RGBMatrix`` (hardware) or ``RGBMatrixEmulator`` depending
+  on the ``EMULATOR`` environment variable.
+* Expose a PIL ``Image``/``ImageDraw`` canvas that plugins draw into, then
+  flush it to the matrix via double-buffering (:meth:`DisplayManager.update_display`).
+* Load and cache TTF/BDF fonts; expose ``draw_text`` for consistent text rendering.
+* Provide ``width`` / ``height`` properties — always use these instead of
+  hard-coding display dimensions.
+* Write periodic PNG snapshots to ``/tmp/led_matrix_preview.png`` for the
+  web-interface live preview.
+* Track scrolling state and gate deferred updates so plugins don't race with
+  an in-progress scroll.
+
+Singleton: only one ``DisplayManager`` instance exists per process.  The
+first call to ``DisplayManager(config)`` creates it; subsequent calls return
+the same object.
+"""
+
 import json
 import os
 import tempfile
@@ -18,6 +43,24 @@ logger = logging.getLogger(__name__)
 logger.setLevel(logging.INFO)  # Set to INFO level
 
 class DisplayManager:
+    """
+    Singleton hardware abstraction layer for the RGB LED matrix.
+
+    Plugins should never interact with ``RGBMatrix`` directly; they use this
+    class to draw content and call :meth:`update_display` to push frames to
+    the panel.
+
+    Typical plugin usage::
+
+        canvas = Image.new('RGB', (self.display_manager.width,
+                                   self.display_manager.height), (0, 0, 0))
+        draw = ImageDraw.Draw(canvas)
+        # ... draw content ...
+        self.display_manager.image = canvas
+        self.display_manager.draw = ImageDraw.Draw(self.display_manager.image)
+        self.display_manager.update_display()
+    """
+
     _instance = None
     _initialized = False
 
@@ -33,6 +76,10 @@ class DisplayManager:
         self._suppress_test_pattern = suppress_test_pattern
         # When True, update_display() and clear() skip hardware writes (used during off-screen content capture)
         self._capture_mode_active = False
+        # Text-width measurement cache: (text, id(font)) -> pixel_width
+        # Avoids re-measuring the same string+font on every display() call.
+        # Cleared on _load_fonts() so stale entries don't survive a font reload.
+        self._text_width_cache: Dict[tuple, int] = {}
         # Snapshot settings for web preview integration (service writes, web reads)
         self._snapshot_path = "/tmp/led_matrix_preview.png"  # nosec B108 - fixed path intentional; web UI reads same path
         self._snapshot_min_interval_sec = 0.2  # max ~5 fps
@@ -437,6 +484,9 @@ class DisplayManager:
 
     def _load_fonts(self):
         """Load fonts with proper error handling."""
+        # Font objects get new id()s after reload, so the text-width cache would
+        # return stale measurements keyed on the old ids.  Clear it here.
+        self._text_width_cache.clear()
         try:
             # Load Press Start 2P font
             self.regular_font = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 8)
@@ -497,22 +547,32 @@ class DisplayManager:
 
 
     def get_text_width(self, text, font):
-        """Get the width of text when rendered with the given font."""
+        """Get the width of text when rendered with the given font.
+
+        Results are cached by (text, font identity) so plugins that measure
+        the same string every frame (e.g. to centre a score) pay only one
+        measurement per unique (text, font) pair.
+        """
+        cache_key = (text, id(font))
+        cached = self._text_width_cache.get(cache_key)
+        if cached is not None:
+            return cached
+
         try:
             if isinstance(font, freetype.Face):
-                # For FreeType faces, calculate width using freetype
                 width = 0
                 for char in text:
                     font.load_char(char)
                     width += font.glyph.advance.x >> 6
-                return width
             else:
-                # For PIL fonts, use textbbox
                 bbox = self.draw.textbbox((0, 0), text, font=font)
-                return bbox[2] - bbox[0]
-        except Exception as e:
-            logger.error(f"Error getting text width: {e}")
-            return 0  # Return 0 as fallback
+                width = bbox[2] - bbox[0]
+        except (AttributeError, TypeError, ValueError, OSError) as e:
+            logger.error("Error getting text width: %s", e)
+            return 0
+
+        self._text_width_cache[cache_key] = width
+        return width
 
     def get_font_height(self, font):
         """Get the height of the given font for line spacing purposes."""