Follow-ups from #441: secret-helper migration, ten more bug fixes, and coverage for every remaining untested module (#444)

* refactor(web): use canonical secret helpers in api_v3; make ConfigManager secret strip/merge array-aware

api_v3.py carried three inline nested copies of find_secret_fields/
separate_secrets (main-config save, plugin-config save, plugin-config
reset). They drifted from each other (one lacked isinstance guards) and
none supported the canonical module's array-item secrets
(accounts[].token). All three endpoints now import from
src/web_interface/secret_helpers.

Adopting the canonical behavior makes array-item secrets reachable, and
their parallel-placeholder shape ([{'token': ...}, {}] alongside the
regular list) was not survivable by ConfigManager's round-trip:
_strip_secrets_recursive dropped the whole key (losing the regular
fields from config.json) and _deep_merge replaced the regular list
wholesale on load. Both are now array-aware:

- strip removes the secret fields from each item and ALWAYS keeps the
  list so indices survive for merge-on-load; whole-key secrets (scalar
  lists, shape mismatches) still drop the key entirely — never leak.
- merge folds each secrets item into the config item at the same index,
  skipping {} placeholders. The regular list's length is authoritative
  in both directions: a user deleting an array item never has it
  resurrected from a stale secrets entry (extras warn and are ignored).

api_v3's own deep_merge intentionally still replaces lists wholesale —
form posts carry complete arrays and index-merging would resurrect
deleted items; a comment now documents that.

Tests: the parity guard flips from 'exactly 3 inline copies' to 'zero,
and the canonical import must exist'; TestArraySecretStripAndMerge
covers the new strip/merge semantics incl. length-mismatch contracts;
new test_api_v3_secret_roundtrip.py drives all three endpoints through
a Flask client with a REAL ConfigManager+SchemaManager over tmp_path,
proving secrets land in config_secrets.json, config.json stays clean,
and a fresh load merges them back into the right array items.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

* fix: repair broken helper paths across display, cache, odds, logging, resolver, repos, config, validator

Nine fixes for bugs surfaced while writing coverage for previously
untested modules (plus the bool-duration quirk pinned in PR #441):

- base_plugin.get_display_duration: exclude bools from both numeric
  branches — display_duration=True no longer reads as a 1-second slot;
  it falls through to config, then the 15.0 default.
- display_helper: draw_error_message/draw_no_data_message called
  _draw_centered_text with the wrong arguments and crashed with
  AttributeError — both now delegate to draw_centered_text.
  draw_scorebug_layout drew status and clock at the same y, overprinting
  each other — they now share one combined top line.
  draw_ticker_layout drew its text starting at x=display_width (fully
  off-canvas), returning a blank frame every time — now draws at x=0;
  scroll_speed stays accepted-but-unused and is documented as such.
- api_helper.clear_cache guarded on a nonexistent CacheManager.clear()
  method, silently never clearing anything; it now uses the real surface
  (clear_cache/delete/list_cache_files) and no-ops safely otherwise.
- base_odds_manager._extract_espn_data raised AttributeError when ESPN
  sent explicit JSON nulls ("homeTeamOdds": null) — every level now
  null-safes with 'or {}'. format_odds_summary gated on
  is_odds_available, which deliberately ignores money lines, so
  ML-only odds formatted as "No odds available" — it now gates only on
  empty/no_odds data and formats money lines.
- logging_config.ContextualFormatter mutated record.msg in place, so a
  second handler prepended the context prefix twice; it now formats a
  copy. log_error hardcoded exc_info=True and raised TypeError when the
  caller passed exc_info — now kwargs.setdefault.
- dynamic_team_resolver wrote its "shared" class cache through self,
  creating instance shadows — the cache was per-instance and every
  scoreboard refetched rankings. Writes now go through the class.
- saved_repositories cleaned URLs with an unanchored .replace('.git','')
  that mangled URLs merely containing '.git' (my.github.io -> myhub.io);
  now strips only a trailing suffix. add/remove also roll back the
  in-memory list when the save fails, so memory always matches disk.
- config_helper.merge_configs shallow-copied the base, aliasing every
  un-overridden nested dict into the result — now deep-copies.
- startup_validator.validate_all accumulated errors/warnings across
  calls — now resets both lists per run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

* test: cover the previously untested modules

Nine new suites plus an extension, asserting the Phase-1b fixed behavior
and pinning the quirks deliberately left alone:

- test_logging_config.py: formatters (JSON shape, no record mutation,
  single prefix through two handlers), PluginLoggerAdapter precedence,
  setup_logging handler hygiene and LEDMATRIX_DEBUG, log_error exc_info.
- test_startup_validator.py: exact messages, error-vs-warning split,
  accessor split (load_config vs get_config), cache-dir branches with
  os.access monkeypatched (root can write anything in CI), idempotence,
  raise_on_errors classification precedence.
- test_config_helper.py (full): load/save round trips, dot-notation
  get/set incl. silent-failure contract, post-fix no-aliasing merge,
  schema validation branches, the '{id}_config' key pin, default-enabled
  pin.
- test_saved_repositories.py: three load shapes, bare-list rewrite pin,
  trailing-only .git strip (my.github.io regression), save-failure
  rollback, type-classification case-sensitivity pin.
- test_api_helper.py: rate-limit math, cache-hit short circuit, ESPN
  URL/key formats, exact User-Agent guard, retry adapter, post-fix
  clear_cache against the real CacheManager surface, ttl-dropped pin.
- test_base_odds_manager.py: cache-key/URL construction, no_odds
  sentinel round trip, stale-cache fallback, null-safe extraction,
  ML-only formatting, is_odds_available truth table (ML-blind by
  contract), config key/attr mismatch pin.
- test_dynamic_team_resolver.py: expansion/dedup/slicing, dropped
  unknown-dynamic names (TOP_ substring hazard pinned), genuinely
  shared class cache (second instance: zero HTTP), TTL expiry,
  failure degradation without raising.
- test_display_helper.py (full): the fixed error/no-data renders,
  combined scorebug top line, non-blank ticker with scroll_speed
  no-op pin, composite upconversion, logo bleed positions, square
  orientation pin.
- test_skin_runtime_cache.py: discovery-cache hit/invalidation
  semantics (manifest mtime, .py edits pinned as non-invalidating),
  sys.modules namespacing contract incl. bare-name restore and stdlib
  shadowing, entry-module execute-once, API minor-version tolerance,
  skin_matches_target table.
- test_sports_capabilities.py (extended): _draw_celebration_layout
  executed for real (flash window, matrix-dims fallback, highlight
  alternation, logo-failure isolation), _should_celebrate_for direct,
  strict duration boundary, score_to_int edges, both-teams-score
  precedence, expired-coalesce refire, disabled-win baseline
  preservation, id-less prune.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

* test: real schedule/dim coverage for DisplayController; fix two vacuous schedule tests

New test_display_controller_schedule.py drives _check_schedule and
_check_dim_schedule on a bare controller stub: same-day and
midnight-crossing windows with inclusive boundaries, global vs per-day vs
legacy-inferred modes (and dim's global-only default — no legacy
inference), per-day disabled days, invalid %H:%M fallbacks, unknown
timezone -> UTC, dim_brightness default 30, inactive-display short
circuit, and the _was_display_active/_was_dimmed transition flags.

test_display_controller.py's test_schedule_disabled and
test_active_hours patched config_service.get_config — which
_check_schedule never reads — so both asserted the init-default value
and could not fail. Rewritten on the test_inactive_hours pattern
(inject controller.config['schedule'], reset the minute gate, flip the
flag to the opposite state first so the assertion has teeth).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

* ci: raise coverage floor to 48%

Measured 50% with the new suites in place (was 47% baseline when the
gate was introduced at 45); floor stays two points under measured.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

* fix: address CodeQL alert and review findings

- config_manager: the "secrets list longer than config list" warning now
  interpolates only config-side data (no key name or secrets-derived
  values), resolving the CodeQL clear-text-logging alert.
- base_plugin: validate_config rejects bool display_duration, matching
  get_display_duration (bool is an int subclass and would otherwise pass
  as a positive number).
- config_helper: merge_configs deep-copies override values in the
  non-recursive branch so mutating the merged result cannot reach back
  into override_config.
- saved_repositories: saves are atomic (temp file + fsync + os.replace),
  so a failed write can no longer truncate saved_repositories.json.
- tests: regression cases for each fix, plus a pin that whole-item
  array secrets (key[] + key[].field both marked) strip to empty {}
  skeletons — no secret values can reach config.json.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh

---------

Co-authored-by: Claude <noreply@anthropic.com>

test/test_config_helper.py

@@ -0,0 +1,253 @@
+"""
+Tests for src/common/config_helper.py — pins the ConfigHelper contract.
+
+Covers: load/save round trips (missing/malformed files return {} rather
+than raising, non-ASCII preserved via ensure_ascii=False, top-level JSON
+lists returned as-is), dot-notation get/set including the silent-failure
+contract when an intermediate key holds a non-dict, merge_configs deep
+semantics with NO aliasing of the base config (the fixed bug — the old
+shallow copy let mutations of the merged result leak into base's nested
+dicts), simplified schema validation including the caught-TypeError path
+when a schema 'type' is given as a string, plugin config key conventions
+('{plugin_id}_config', enabled defaults True), and required-key checks
+where a key present with value None counts as present.
+"""
+
+import json
+
+import pytest
+
+from src.common.config_helper import ConfigHelper
+
+
+@pytest.fixture
+def helper():
+    return ConfigHelper()
+
+
+class TestLoadConfig:
+    def test_missing_file_returns_empty_dict(self, helper, tmp_path):
+        assert helper.load_config(tmp_path / "nope.json") == {}
+
+    def test_malformed_json_returns_empty_dict(self, helper, tmp_path):
+        path = tmp_path / "bad.json"
+        path.write_text("{ this is not json", encoding="utf-8")
+        assert helper.load_config(path) == {}
+
+    def test_top_level_list_returned_as_is(self, helper, tmp_path):
+        # load_config does not enforce a dict shape: a JSON list comes
+        # straight back. Pinned as a characterization of current behavior.
+        path = tmp_path / "list.json"
+        path.write_text("[1, 2, 3]", encoding="utf-8")
+        assert helper.load_config(path) == [1, 2, 3]
+
+
+class TestSaveConfig:
+    def test_round_trip(self, helper, tmp_path):
+        path = tmp_path / "config.json"
+        config = {'display': {'hardware': {'rows': 32}}, 'timezone': 'UTC'}
+        assert helper.save_config(config, path) is True
+        assert helper.load_config(path) == config
+
+    def test_creates_parent_directories(self, helper, tmp_path):
+        path = tmp_path / "deep" / "nested" / "config.json"
+        assert helper.save_config({'a': 1}, path) is True
+        assert path.exists()
+        assert helper.load_config(path) == {'a': 1}
+
+    def test_non_ascii_survives_round_trip(self, helper, tmp_path):
+        path = tmp_path / "config.json"
+        config = {'city': 'Zürich', 'note': 'météo ☀'}
+        assert helper.save_config(config, path) is True
+        assert helper.load_config(path) == config
+        # ensure_ascii=False: characters are written raw, not \u-escaped
+        assert 'Zürich' in path.read_text(encoding='utf-8')
+
+    def test_directory_path_returns_false_not_raise(self, helper, tmp_path):
+        assert helper.save_config({'a': 1}, tmp_path) is False
+
+
+class TestGetConfigValue:
+    def test_dot_notation_hit(self, helper):
+        config = {'display': {'hardware': {'rows': 32}}}
+        assert helper.get_config_value(config, 'display.hardware.rows') == 32
+
+    def test_missing_returns_default(self, helper):
+        sentinel = object()
+        assert helper.get_config_value({}, 'display.rows', default=sentinel) is sentinel
+
+    def test_intermediate_non_dict_returns_default(self, helper):
+        config = {'display': 'not-a-dict'}
+        assert helper.get_config_value(config, 'display.hardware.rows', default=64) == 64
+
+    def test_required_missing_raises_keyerror(self, helper):
+        with pytest.raises(KeyError):
+            helper.get_config_value({}, 'display.rows', required=True)
+
+
+class TestSetConfigValue:
+    def test_sets_top_level(self, helper):
+        config = {}
+        helper.set_config_value(config, 'timezone', 'UTC')
+        assert config == {'timezone': 'UTC'}
+
+    def test_auto_creates_intermediates(self, helper):
+        config = {}
+        helper.set_config_value(config, 'display.hardware.rows', 32)
+        assert config == {'display': {'hardware': {'rows': 32}}}
+
+    def test_silent_failure_on_non_dict_intermediate(self, helper):
+        # 'a' exists but holds an int; the assignment attempt raises
+        # TypeError internally, which set_config_value swallows and logs.
+        # The config is left unchanged — pinned silent-failure contract.
+        config = {'a': 5}
+        helper.set_config_value(config, 'a.b', 1)
+        assert config == {'a': 5}
+
+
+class TestMergeConfigs:
+    def test_nested_dicts_merge_recursively(self, helper):
+        base = {'display': {'rows': 32, 'cols': 64}, 'timezone': 'UTC'}
+        override = {'display': {'cols': 128, 'brightness': 90}}
+        merged = helper.merge_configs(base, override)
+        assert merged == {
+            'display': {'rows': 32, 'cols': 128, 'brightness': 90},
+            'timezone': 'UTC',
+        }
+
+    def test_scalar_override_wins_over_dict(self, helper):
+        merged = helper.merge_configs({'display': {'rows': 32}}, {'display': 7})
+        assert merged['display'] == 7
+
+    def test_dict_override_wins_over_scalar(self, helper):
+        merged = helper.merge_configs({'display': 7}, {'display': {'rows': 32}})
+        assert merged['display'] == {'rows': 32}
+
+    def test_no_aliasing_of_base(self, helper):
+        # Post-fix: merge deep-copies base, so mutating the result never
+        # leaks back into the caller's base config.
+        base = {'display': {'x': 1}}
+        merged = helper.merge_configs(base, {})
+        assert merged['display'] is not base['display']
+        merged['display']['x'] = 99
+        assert base['display']['x'] == 1
+
+    def test_inputs_unchanged(self, helper):
+        base = {'a': {'b': 1}}
+        override = {'a': {'c': 2}}
+        helper.merge_configs(base, override)
+        assert base == {'a': {'b': 1}}
+        assert override == {'a': {'c': 2}}
+
+    def test_no_aliasing_of_override_values(self, helper):
+        # The non-recursive branch must deep-copy the override value too:
+        # mutating a merged-in list or dict must not reach back into
+        # override_config.
+        override = {'teams': ['A', 'B'], 'nested': {'x': [1]}}
+        merged = helper.merge_configs({}, override)
+        merged['teams'].append('C')
+        merged['nested']['x'].append(2)
+        assert override == {'teams': ['A', 'B'], 'nested': {'x': [1]}}
+
+
+class TestValidateConfig:
+    def test_no_schema_dict_is_valid(self, helper):
+        assert helper.validate_config({'a': 1}) is True
+
+    def test_no_schema_list_is_invalid(self, helper):
+        assert helper.validate_config([1, 2]) is False
+
+    def test_required_key_missing_is_invalid(self, helper):
+        schema = {'rows': {'required': True, 'type': int}}
+        assert helper.validate_config({}, schema) is False
+
+    def test_optional_key_missing_is_valid(self, helper):
+        schema = {'rows': {'required': False, 'type': int}}
+        assert helper.validate_config({}, schema) is True
+
+    def test_wrong_type_is_invalid(self, helper):
+        schema = {'rows': {'type': int}}
+        assert helper.validate_config({'rows': 'thirty-two'}, schema) is False
+        assert helper.validate_config({'rows': 32}, schema) is True
+
+    def test_allowed_values_violation_is_invalid(self, helper):
+        schema = {'mode': {'allowed_values': ['clock', 'weather']}}
+        assert helper.validate_config({'mode': 'stocks'}, schema) is False
+        assert helper.validate_config({'mode': 'clock'}, schema) is True
+
+    def test_string_type_in_schema_is_invalid_via_typeerror(self, helper):
+        # 'type' given as the STRING "int" makes isinstance() raise
+        # TypeError; validate_config catches it and returns False rather
+        # than raising. Pinned characterization.
+        schema = {'rows': {'type': 'int'}}
+        assert helper.validate_config({'rows': 32}, schema) is False
+
+
+class TestPluginConfigHelpers:
+    def test_get_plugin_config_uses_suffixed_key(self, helper):
+        plugin_cfg = {'enabled': True, 'display_duration': 30}
+        assert helper.get_plugin_config({'clock_config': plugin_cfg}, 'clock') == plugin_cfg
+
+    def test_get_plugin_config_bare_id_key_not_found(self, helper):
+        # Only '{plugin_id}_config' is consulted — a bare 'clock' section
+        # is invisible to this helper. Pinned key contract.
+        assert helper.get_plugin_config({'clock': {'enabled': True}}, 'clock') == {}
+
+    def test_create_default_config_wraps_in_suffixed_key(self, helper):
+        defaults = {'enabled': True}
+        assert helper.create_default_config('clock', defaults) == {'clock_config': defaults}
+
+    def test_is_plugin_enabled_defaults_true_for_unknown(self, helper):
+        assert helper.is_plugin_enabled({}, 'clock') is True
+
+    def test_is_plugin_enabled_false_when_disabled(self, helper):
+        config = {'clock_config': {'enabled': False}}
+        assert helper.is_plugin_enabled(config, 'clock') is False
+
+    def test_is_plugin_enabled_ignores_bare_id_key(self, helper):
+        # Disabled under the wrong key -> still reported enabled (default).
+        config = {'clock': {'enabled': False}}
+        assert helper.is_plugin_enabled(config, 'clock') is True
+
+
+class TestSportsAndDisplayHelpers:
+    def test_get_display_config(self, helper):
+        display = {'hardware': {'rows': 32}}
+        assert helper.get_display_config({'display': display}) == display
+        assert helper.get_display_config({}) == {}
+
+    def test_get_sports_config_uses_scoreboard_suffix(self, helper):
+        sport_cfg = {'favorite_teams': ['TB']}
+        config = {'football_scoreboard': sport_cfg}
+        assert helper.get_sports_config(config, 'football') == sport_cfg
+        assert helper.get_sports_config(config, 'hockey') == {}
+
+    def test_get_favorite_teams(self, helper):
+        config = {'football_scoreboard': {'favorite_teams': ['TB', 'DAL']}}
+        assert helper.get_favorite_teams(config, 'football') == ['TB', 'DAL']
+        assert helper.get_favorite_teams({}, 'football') == []
+
+    def test_get_display_modes(self, helper):
+        modes = {'live': True, 'recent': False}
+        config = {'football_scoreboard': {'display_modes': modes}}
+        assert helper.get_display_modes(config, 'football') == modes
+        assert helper.get_display_modes({}, 'football') == {}
+
+
+class TestValidateRequiredKeys:
+    def test_returns_missing_subset(self, helper):
+        config = {'a': 1, 'c': {'d': 2}}
+        missing = helper.validate_required_keys(config, ['a', 'b', 'c.d', 'c.e'])
+        assert missing == ['b', 'c.e']
+
+    def test_dot_notation_present(self, helper):
+        config = {'display': {'hardware': {'rows': 32}}}
+        assert helper.validate_required_keys(config, ['display.hardware.rows']) == []
+
+    def test_empty_requirements(self, helper):
+        assert helper.validate_required_keys({'a': 1}, []) == []
+
+    def test_present_with_none_counts_as_present(self, helper):
+        # _has_key checks key membership, not truthiness — a key set to
+        # None is NOT reported missing. Pinned semantics.
+        assert helper.validate_required_keys({'a': None}, ['a']) == []