mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-08-08 04:08:06 +00:00
* refactor(web): use canonical secret helpers in api_v3; make ConfigManager secret strip/merge array-aware
api_v3.py carried three inline nested copies of find_secret_fields/
separate_secrets (main-config save, plugin-config save, plugin-config
reset). They drifted from each other (one lacked isinstance guards) and
none supported the canonical module's array-item secrets
(accounts[].token). All three endpoints now import from
src/web_interface/secret_helpers.
Adopting the canonical behavior makes array-item secrets reachable, and
their parallel-placeholder shape ([{'token': ...}, {}] alongside the
regular list) was not survivable by ConfigManager's round-trip:
_strip_secrets_recursive dropped the whole key (losing the regular
fields from config.json) and _deep_merge replaced the regular list
wholesale on load. Both are now array-aware:
- strip removes the secret fields from each item and ALWAYS keeps the
list so indices survive for merge-on-load; whole-key secrets (scalar
lists, shape mismatches) still drop the key entirely — never leak.
- merge folds each secrets item into the config item at the same index,
skipping {} placeholders. The regular list's length is authoritative
in both directions: a user deleting an array item never has it
resurrected from a stale secrets entry (extras warn and are ignored).
api_v3's own deep_merge intentionally still replaces lists wholesale —
form posts carry complete arrays and index-merging would resurrect
deleted items; a comment now documents that.
Tests: the parity guard flips from 'exactly 3 inline copies' to 'zero,
and the canonical import must exist'; TestArraySecretStripAndMerge
covers the new strip/merge semantics incl. length-mismatch contracts;
new test_api_v3_secret_roundtrip.py drives all three endpoints through
a Flask client with a REAL ConfigManager+SchemaManager over tmp_path,
proving secrets land in config_secrets.json, config.json stays clean,
and a fresh load merges them back into the right array items.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh
* fix: repair broken helper paths across display, cache, odds, logging, resolver, repos, config, validator
Nine fixes for bugs surfaced while writing coverage for previously
untested modules (plus the bool-duration quirk pinned in PR #441):
- base_plugin.get_display_duration: exclude bools from both numeric
branches — display_duration=True no longer reads as a 1-second slot;
it falls through to config, then the 15.0 default.
- display_helper: draw_error_message/draw_no_data_message called
_draw_centered_text with the wrong arguments and crashed with
AttributeError — both now delegate to draw_centered_text.
draw_scorebug_layout drew status and clock at the same y, overprinting
each other — they now share one combined top line.
draw_ticker_layout drew its text starting at x=display_width (fully
off-canvas), returning a blank frame every time — now draws at x=0;
scroll_speed stays accepted-but-unused and is documented as such.
- api_helper.clear_cache guarded on a nonexistent CacheManager.clear()
method, silently never clearing anything; it now uses the real surface
(clear_cache/delete/list_cache_files) and no-ops safely otherwise.
- base_odds_manager._extract_espn_data raised AttributeError when ESPN
sent explicit JSON nulls ("homeTeamOdds": null) — every level now
null-safes with 'or {}'. format_odds_summary gated on
is_odds_available, which deliberately ignores money lines, so
ML-only odds formatted as "No odds available" — it now gates only on
empty/no_odds data and formats money lines.
- logging_config.ContextualFormatter mutated record.msg in place, so a
second handler prepended the context prefix twice; it now formats a
copy. log_error hardcoded exc_info=True and raised TypeError when the
caller passed exc_info — now kwargs.setdefault.
- dynamic_team_resolver wrote its "shared" class cache through self,
creating instance shadows — the cache was per-instance and every
scoreboard refetched rankings. Writes now go through the class.
- saved_repositories cleaned URLs with an unanchored .replace('.git','')
that mangled URLs merely containing '.git' (my.github.io -> myhub.io);
now strips only a trailing suffix. add/remove also roll back the
in-memory list when the save fails, so memory always matches disk.
- config_helper.merge_configs shallow-copied the base, aliasing every
un-overridden nested dict into the result — now deep-copies.
- startup_validator.validate_all accumulated errors/warnings across
calls — now resets both lists per run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh
* test: cover the previously untested modules
Nine new suites plus an extension, asserting the Phase-1b fixed behavior
and pinning the quirks deliberately left alone:
- test_logging_config.py: formatters (JSON shape, no record mutation,
single prefix through two handlers), PluginLoggerAdapter precedence,
setup_logging handler hygiene and LEDMATRIX_DEBUG, log_error exc_info.
- test_startup_validator.py: exact messages, error-vs-warning split,
accessor split (load_config vs get_config), cache-dir branches with
os.access monkeypatched (root can write anything in CI), idempotence,
raise_on_errors classification precedence.
- test_config_helper.py (full): load/save round trips, dot-notation
get/set incl. silent-failure contract, post-fix no-aliasing merge,
schema validation branches, the '{id}_config' key pin, default-enabled
pin.
- test_saved_repositories.py: three load shapes, bare-list rewrite pin,
trailing-only .git strip (my.github.io regression), save-failure
rollback, type-classification case-sensitivity pin.
- test_api_helper.py: rate-limit math, cache-hit short circuit, ESPN
URL/key formats, exact User-Agent guard, retry adapter, post-fix
clear_cache against the real CacheManager surface, ttl-dropped pin.
- test_base_odds_manager.py: cache-key/URL construction, no_odds
sentinel round trip, stale-cache fallback, null-safe extraction,
ML-only formatting, is_odds_available truth table (ML-blind by
contract), config key/attr mismatch pin.
- test_dynamic_team_resolver.py: expansion/dedup/slicing, dropped
unknown-dynamic names (TOP_ substring hazard pinned), genuinely
shared class cache (second instance: zero HTTP), TTL expiry,
failure degradation without raising.
- test_display_helper.py (full): the fixed error/no-data renders,
combined scorebug top line, non-blank ticker with scroll_speed
no-op pin, composite upconversion, logo bleed positions, square
orientation pin.
- test_skin_runtime_cache.py: discovery-cache hit/invalidation
semantics (manifest mtime, .py edits pinned as non-invalidating),
sys.modules namespacing contract incl. bare-name restore and stdlib
shadowing, entry-module execute-once, API minor-version tolerance,
skin_matches_target table.
- test_sports_capabilities.py (extended): _draw_celebration_layout
executed for real (flash window, matrix-dims fallback, highlight
alternation, logo-failure isolation), _should_celebrate_for direct,
strict duration boundary, score_to_int edges, both-teams-score
precedence, expired-coalesce refire, disabled-win baseline
preservation, id-less prune.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh
* test: real schedule/dim coverage for DisplayController; fix two vacuous schedule tests
New test_display_controller_schedule.py drives _check_schedule and
_check_dim_schedule on a bare controller stub: same-day and
midnight-crossing windows with inclusive boundaries, global vs per-day vs
legacy-inferred modes (and dim's global-only default — no legacy
inference), per-day disabled days, invalid %H:%M fallbacks, unknown
timezone -> UTC, dim_brightness default 30, inactive-display short
circuit, and the _was_display_active/_was_dimmed transition flags.
test_display_controller.py's test_schedule_disabled and
test_active_hours patched config_service.get_config — which
_check_schedule never reads — so both asserted the init-default value
and could not fail. Rewritten on the test_inactive_hours pattern
(inject controller.config['schedule'], reset the minute gate, flip the
flag to the opposite state first so the assertion has teeth).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh
* ci: raise coverage floor to 48%
Measured 50% with the new suites in place (was 47% baseline when the
gate was introduced at 45); floor stays two points under measured.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh
* fix: address CodeQL alert and review findings
- config_manager: the "secrets list longer than config list" warning now
interpolates only config-side data (no key name or secrets-derived
values), resolving the CodeQL clear-text-logging alert.
- base_plugin: validate_config rejects bool display_duration, matching
get_display_duration (bool is an int subclass and would otherwise pass
as a positive number).
- config_helper: merge_configs deep-copies override values in the
non-recursive branch so mutating the merged result cannot reach back
into override_config.
- saved_repositories: saves are atomic (temp file + fsync + os.replace),
so a failed write can no longer truncate saved_repositories.json.
- tests: regression cases for each fix, plus a pin that whole-item
array secrets (key[] + key[].field both marked) strip to empty {}
skeletons — no secret values can reach config.json.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NohXi78cwsAKtN1sCfxjUh
---------
Co-authored-by: Claude <noreply@anthropic.com>
362 lines
12 KiB
Python
362 lines
12 KiB
Python
"""
|
|
Config Helper
|
|
|
|
Handles configuration management and validation for LED matrix plugins.
|
|
Extracted from LEDMatrix core to provide reusable functionality for plugins.
|
|
"""
|
|
|
|
import copy
|
|
import json
|
|
import logging
|
|
from pathlib import Path
|
|
from typing import Any, Dict, List, Optional, Union
|
|
|
|
|
|
class ConfigHelper:
|
|
"""
|
|
Helper class for configuration management and validation.
|
|
|
|
Provides functionality for:
|
|
- Loading and saving configuration files
|
|
- Validating configuration against schemas
|
|
- Merging configurations
|
|
- Getting configuration values with defaults
|
|
- Configuration schema validation
|
|
"""
|
|
|
|
def __init__(self, logger: Optional[logging.Logger] = None):
|
|
"""
|
|
Initialize the ConfigHelper.
|
|
|
|
Args:
|
|
logger: Optional logger instance
|
|
"""
|
|
self.logger = logger or logging.getLogger(__name__)
|
|
|
|
def load_config(self, config_path: Union[str, Path]) -> Dict[str, Any]:
|
|
"""
|
|
Load configuration from a JSON file.
|
|
|
|
Args:
|
|
config_path: Path to configuration file
|
|
|
|
Returns:
|
|
Configuration dictionary
|
|
"""
|
|
config_path = Path(config_path)
|
|
|
|
try:
|
|
if not config_path.exists():
|
|
self.logger.warning(f"Configuration file not found: {config_path}")
|
|
return {}
|
|
|
|
with open(config_path, 'r', encoding='utf-8') as f:
|
|
config = json.load(f)
|
|
|
|
self.logger.debug(f"Loaded configuration from {config_path}")
|
|
return config
|
|
|
|
except json.JSONDecodeError as e:
|
|
self.logger.error(f"Invalid JSON in configuration file {config_path}: {e}")
|
|
return {}
|
|
except Exception as e:
|
|
self.logger.error(f"Error loading configuration from {config_path}: {e}")
|
|
return {}
|
|
|
|
def save_config(self, config: Dict[str, Any], config_path: Union[str, Path]) -> bool:
|
|
"""
|
|
Save configuration to a JSON file.
|
|
|
|
Args:
|
|
config: Configuration dictionary to save
|
|
config_path: Path to save configuration file
|
|
|
|
Returns:
|
|
True if successful, False otherwise
|
|
"""
|
|
config_path = Path(config_path)
|
|
|
|
try:
|
|
# Ensure directory exists
|
|
config_path.parent.mkdir(parents=True, exist_ok=True)
|
|
|
|
with open(config_path, 'w', encoding='utf-8') as f:
|
|
json.dump(config, f, indent=2, ensure_ascii=False)
|
|
|
|
self.logger.debug(f"Saved configuration to {config_path}")
|
|
return True
|
|
|
|
except Exception as e:
|
|
self.logger.error(f"Error saving configuration to {config_path}: {e}")
|
|
return False
|
|
|
|
def get_config_value(self, config: Dict[str, Any], key: str,
|
|
default: Any = None, required: bool = False) -> Any:
|
|
"""
|
|
Get a configuration value with optional default.
|
|
|
|
Args:
|
|
config: Configuration dictionary
|
|
key: Configuration key (supports dot notation like 'display.width')
|
|
default: Default value if key not found
|
|
required: If True, raise error if key not found
|
|
|
|
Returns:
|
|
Configuration value or default
|
|
"""
|
|
try:
|
|
# Support dot notation for nested keys
|
|
keys = key.split('.')
|
|
value = config
|
|
|
|
for k in keys:
|
|
if isinstance(value, dict) and k in value:
|
|
value = value[k]
|
|
else:
|
|
if required:
|
|
raise KeyError(f"Required configuration key not found: {key}")
|
|
return default
|
|
|
|
return value
|
|
|
|
except Exception as e:
|
|
if required:
|
|
raise
|
|
self.logger.warning(f"Error getting config value for {key}: {e}")
|
|
return default
|
|
|
|
def set_config_value(self, config: Dict[str, Any], key: str, value: Any) -> None:
|
|
"""
|
|
Set a configuration value.
|
|
|
|
Args:
|
|
config: Configuration dictionary to modify
|
|
key: Configuration key (supports dot notation)
|
|
value: Value to set
|
|
"""
|
|
try:
|
|
# Support dot notation for nested keys
|
|
keys = key.split('.')
|
|
current = config
|
|
|
|
# Navigate to parent of target key
|
|
for k in keys[:-1]:
|
|
if k not in current:
|
|
current[k] = {}
|
|
current = current[k]
|
|
|
|
# Set the value
|
|
current[keys[-1]] = value
|
|
|
|
except Exception as e:
|
|
self.logger.error(f"Error setting config value for {key}: {e}")
|
|
|
|
def merge_configs(self, base_config: Dict[str, Any],
|
|
override_config: Dict[str, Any]) -> Dict[str, Any]:
|
|
"""
|
|
Merge two configuration dictionaries.
|
|
|
|
Args:
|
|
base_config: Base configuration
|
|
override_config: Configuration to merge in (takes precedence)
|
|
|
|
Returns:
|
|
Merged configuration dictionary (fully independent of both
|
|
inputs — a shallow copy would alias un-overridden nested dicts,
|
|
so mutating the result would mutate the caller's base config).
|
|
"""
|
|
merged = copy.deepcopy(base_config)
|
|
|
|
for key, value in override_config.items():
|
|
if key in merged and isinstance(merged[key], dict) and isinstance(value, dict):
|
|
# Recursively merge nested dictionaries
|
|
merged[key] = self.merge_configs(merged[key], value)
|
|
else:
|
|
# Override with new value — deep-copied so mutating the
|
|
# merged result can't reach back into override_config.
|
|
merged[key] = copy.deepcopy(value)
|
|
|
|
return merged
|
|
|
|
def validate_config(self, config: Dict[str, Any],
|
|
schema: Optional[Dict[str, Any]] = None) -> bool:
|
|
"""
|
|
Validate configuration against a schema.
|
|
|
|
Args:
|
|
config: Configuration to validate
|
|
schema: Validation schema (optional)
|
|
|
|
Returns:
|
|
True if valid, False otherwise
|
|
"""
|
|
if schema is None:
|
|
# Basic validation - just check if it's a dictionary
|
|
return isinstance(config, dict)
|
|
|
|
try:
|
|
return self._validate_against_schema(config, schema)
|
|
except Exception as e:
|
|
self.logger.error(f"Configuration validation error: {e}")
|
|
return False
|
|
|
|
def get_plugin_config(self, config: Dict[str, Any], plugin_id: str) -> Dict[str, Any]:
|
|
"""
|
|
Get plugin-specific configuration.
|
|
|
|
Args:
|
|
config: Full configuration dictionary
|
|
plugin_id: Plugin identifier
|
|
|
|
Returns:
|
|
Plugin-specific configuration
|
|
"""
|
|
plugin_key = f"{plugin_id}_config"
|
|
return config.get(plugin_key, {})
|
|
|
|
def create_default_config(self, plugin_id: str,
|
|
default_values: Dict[str, Any]) -> Dict[str, Any]:
|
|
"""
|
|
Create a default configuration for a plugin.
|
|
|
|
Args:
|
|
plugin_id: Plugin identifier
|
|
default_values: Default configuration values
|
|
|
|
Returns:
|
|
Default configuration dictionary
|
|
"""
|
|
return {
|
|
f"{plugin_id}_config": default_values
|
|
}
|
|
|
|
def validate_required_keys(self, config: Dict[str, Any],
|
|
required_keys: List[str]) -> List[str]:
|
|
"""
|
|
Validate that required keys are present in configuration.
|
|
|
|
Args:
|
|
config: Configuration to validate
|
|
required_keys: List of required keys
|
|
|
|
Returns:
|
|
List of missing keys
|
|
"""
|
|
missing_keys = []
|
|
|
|
for key in required_keys:
|
|
if not self._has_key(config, key):
|
|
missing_keys.append(key)
|
|
|
|
return missing_keys
|
|
|
|
def get_display_config(self, config: Dict[str, Any]) -> Dict[str, Any]:
|
|
"""
|
|
Get display-related configuration.
|
|
|
|
Args:
|
|
config: Full configuration dictionary
|
|
|
|
Returns:
|
|
Display configuration
|
|
"""
|
|
return config.get('display', {})
|
|
|
|
def get_sports_config(self, config: Dict[str, Any], sport: str) -> Dict[str, Any]:
|
|
"""
|
|
Get sport-specific configuration.
|
|
|
|
Args:
|
|
config: Full configuration dictionary
|
|
sport: Sport name (e.g., 'basketball', 'football')
|
|
|
|
Returns:
|
|
Sport-specific configuration
|
|
"""
|
|
return config.get(f"{sport}_scoreboard", {})
|
|
|
|
def is_plugin_enabled(self, config: Dict[str, Any], plugin_id: str) -> bool:
|
|
"""
|
|
Check if a plugin is enabled.
|
|
|
|
Args:
|
|
config: Full configuration dictionary
|
|
plugin_id: Plugin identifier
|
|
|
|
Returns:
|
|
True if plugin is enabled
|
|
"""
|
|
plugin_config = self.get_plugin_config(config, plugin_id)
|
|
return plugin_config.get('enabled', True)
|
|
|
|
def get_favorite_teams(self, config: Dict[str, Any], sport: str) -> List[str]:
|
|
"""
|
|
Get favorite teams for a sport.
|
|
|
|
Args:
|
|
config: Full configuration dictionary
|
|
sport: Sport name
|
|
|
|
Returns:
|
|
List of favorite team abbreviations
|
|
"""
|
|
sport_config = self.get_sports_config(config, sport)
|
|
return sport_config.get('favorite_teams', [])
|
|
|
|
def get_display_modes(self, config: Dict[str, Any], sport: str) -> Dict[str, bool]:
|
|
"""
|
|
Get display modes for a sport.
|
|
|
|
Args:
|
|
config: Full configuration dictionary
|
|
sport: Sport name
|
|
|
|
Returns:
|
|
Dictionary of display modes and their enabled status
|
|
"""
|
|
sport_config = self.get_sports_config(config, sport)
|
|
return sport_config.get('display_modes', {})
|
|
|
|
def _validate_against_schema(self, config: Dict[str, Any],
|
|
schema: Dict[str, Any]) -> bool:
|
|
"""Validate configuration against a schema."""
|
|
# This is a simplified schema validation
|
|
# In a real implementation, you might use a library like jsonschema
|
|
|
|
for key, schema_info in schema.items():
|
|
if key not in config:
|
|
if schema_info.get('required', False):
|
|
self.logger.error(f"Missing required configuration key: {key}")
|
|
return False
|
|
continue
|
|
|
|
value = config[key]
|
|
expected_type = schema_info.get('type')
|
|
|
|
if expected_type and not isinstance(value, expected_type):
|
|
self.logger.error(f"Configuration key {key} has wrong type. Expected {expected_type}, got {type(value)}")
|
|
return False
|
|
|
|
# Validate allowed values
|
|
allowed_values = schema_info.get('allowed_values')
|
|
if allowed_values and value not in allowed_values:
|
|
self.logger.error(f"Configuration key {key} has invalid value: {value}. Allowed: {allowed_values}")
|
|
return False
|
|
|
|
return True
|
|
|
|
def _has_key(self, config: Dict[str, Any], key: str) -> bool:
|
|
"""Check if a key exists in configuration (supports dot notation)."""
|
|
try:
|
|
keys = key.split('.')
|
|
current = config
|
|
|
|
for k in keys:
|
|
if not isinstance(current, dict) or k not in current:
|
|
return False
|
|
current = current[k]
|
|
|
|
return True
|
|
except Exception:
|
|
return False
|