mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-05 23:05:10 +00:00
* refactor(cache): collapse CacheStrategy's all-60 defaults table and twin soccer branch get_sport_live_interval() without a config manager looked the sport up in a table where every value was 60, with 60 as the fallback; it now returns 60. get_data_type_from_key() had an `if 'soccer'` branch returning the same 'sports_live' as its else. test_cache_strategy_intervals pins the returned strategy for every data type x sport key x config-manager shape; it passes unchanged on the old code. A 2,544-entry dump of every CacheStrategy method over a wider grid is identical before and after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(cache): drop CacheStrategy's `<sport>_scoreboard` config lookup get_sport_live_interval() and get_cache_strategy() read live/recent/ upcoming intervals from config[f"{sport}_scoreboard"]. Those sections belonged to the built-in scoreboards the plugin system replaced; plugin config is keyed by plugin id ("football-scoreboard"), so on a current config the lookup always fell through to the defaults (60 live, 1800 recent, 10800 upcoming), which are now returned directly. The one input where this differs: a config.json upgraded from the pre-plugin era that still carries e.g. an "nfl_scoreboard" section (no code removes them), queried with an explicit sport key. No caller in core or the plugin monorepo passes a sport key here -- get_with_auto_strategy only derives one for keys classed sports_live/live_scores, and its callers (odds managers, odds-ticker) use odds keys -- so the stale section was unreachable in practice. A dump of every CacheStrategy method over 2,544 inputs differs from the previous commit only in those 45 legacy-config entries; the test grid now includes that shape. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(cache): list cache files without holding the memory-tier lock CacheManager.list_cache_files() held the in-memory cache's lock while it listed and stat'd the whole cache directory -- 8,864 files on a real rig -- so every get()/set() from the display loop and plugins waited out the scan. The lock never protected the disk: DiskCache writes and deletes under their own lock, and a file vanishing between listdir and stat was already handled (logged and skipped). The body is unchanged apart from the dedent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(cache): delegate memory-tier cleanup and stats to MemoryCache CacheManager._cleanup_memory_cache() was a line-for-line copy of MemoryCache.cleanup(), and get_memory_cache_stats() a copy of MemoryCache.get_stats(), both reaching into the component's private _cache/_timestamps/_lock through "backward compatibility" aliases bound in __init__. So the component's own cleanup and stats only ever ran in tests, and the aliases went stale whenever the component was swapped (test_cache_ttl_honoured does). Both now delegate, and the aliases are gone: nothing in core, the tests, or the ledmatrix-plugins monorepo reads them. Behaviour is the same. Compared line by line, the two cleanups differ only in the sort key's fallback (0 vs 0.0, which orders identically), range+bounds check vs slice for the eviction, and the logger name on the DEBUG summary line (src.cache_manager -> src.cache.memory_cache). A differential run over 20,000 random memory states (str/None/garbage/ future timestamps, orphan keys, sizes 0-12, forced and throttled runs) gives identical removed counts, resulting dicts and last-cleanup times; the same harness catches each of three seeded mutations of MemoryCache.cleanup. The throttle clock also moves with it: CacheManager kept its own copy of last-cleanup, the component's is used now, and they started equal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(background): inline the sport cache key and drop the unused request queue get_sport_cache_key() constructed a whole CacheManager -- ConfigManager, config parse, cache-dir probing with test-file writes -- to return f"{sport}_{date}". It now builds the key itself in the same format as CacheManager.generate_sport_cache_key() (UTC date, %Y%m%d); tests check the two agree for explicit dates and, with a frozen clock at 03:30 UTC, for the default date. Median per call on Windows: ~0.6 ms -> ~2 us (alternating runs); on a Pi the old path also wrote a probe file per call. request_queue was a PriorityQueue nothing ever put into: requests go straight to the executor, so `priority` never did anything. The queue is gone; the `priority` parameter and FetchRequest field stay (every monorepo scoreboard passes priority=) and are documented as ignored, and get_statistics() keeps reporting queue_size, now a literal 0 as it always was in practice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
266 lines
8.9 KiB
Python
266 lines
8.9 KiB
Python
"""
|
|
Cache Strategy
|
|
|
|
Manages cache strategies (TTLs) for different data types.
|
|
"""
|
|
|
|
import logging
|
|
from typing import Dict, Any, Optional
|
|
from datetime import datetime
|
|
import pytz
|
|
|
|
|
|
class CacheStrategy:
|
|
"""Manages cache strategies for different data types."""
|
|
|
|
def __init__(self, config_manager: Optional[Any] = None, logger: Optional[logging.Logger] = None) -> None:
|
|
"""
|
|
Initialize cache strategy manager.
|
|
|
|
Args:
|
|
config_manager: Optional ConfigManager instance. Kept for callers
|
|
that pass one; no strategy currently reads it.
|
|
logger: Optional logger instance
|
|
"""
|
|
self.config_manager = config_manager
|
|
self.logger = logger or logging.getLogger(__name__)
|
|
|
|
def get_sport_live_interval(self, sport_key: str) -> int:
|
|
"""
|
|
Live-data cache interval, in seconds, for a sport: 60 for every sport.
|
|
|
|
This used to read ``live_update_interval`` from a ``<sport>_scoreboard``
|
|
config section. Those sections belonged to the built-in scoreboards
|
|
that the plugin system replaced; plugin config is keyed by plugin id
|
|
(``football-scoreboard``), so the lookup always fell back to 60.
|
|
|
|
Args:
|
|
sport_key: Sport identifier (e.g., 'nba', 'nfl')
|
|
|
|
Returns:
|
|
Live update interval in seconds
|
|
"""
|
|
return 60
|
|
|
|
def get_cache_strategy(self, data_type: str, sport_key: Optional[str] = None) -> Dict[str, Any]:
|
|
"""
|
|
Get cache strategy for different data types.
|
|
|
|
Args:
|
|
data_type: Type of data (e.g., 'live_scores', 'stocks', 'weather_current')
|
|
sport_key: Optional sport key; for live data it selects the
|
|
per-sport interval from :meth:`get_sport_live_interval`
|
|
instead of the generic live default.
|
|
|
|
Returns:
|
|
Dictionary with cache strategy (max_age, memory_ttl, etc.)
|
|
"""
|
|
live_interval = None
|
|
if sport_key and data_type in ['sports_live', 'live_scores']:
|
|
live_interval = self.get_sport_live_interval(sport_key)
|
|
|
|
strategies = {
|
|
# Ultra time-sensitive data (live scores, current weather)
|
|
'live_scores': {
|
|
'max_age': live_interval or 15, # Use sport-specific interval
|
|
'memory_ttl': (live_interval or 15) * 2, # 2x for memory cache
|
|
'force_refresh': True
|
|
},
|
|
'sports_live': {
|
|
'max_age': live_interval or 30, # Use sport-specific interval
|
|
'memory_ttl': (live_interval or 30) * 2,
|
|
'force_refresh': True
|
|
},
|
|
'weather_current': {
|
|
'max_age': 300, # 5 minutes
|
|
'memory_ttl': 600,
|
|
'force_refresh': False
|
|
},
|
|
|
|
# Market data (stocks, crypto)
|
|
'stocks': {
|
|
'max_age': 600, # 10 minutes
|
|
'memory_ttl': 1200,
|
|
'market_hours_only': True,
|
|
'force_refresh': False
|
|
},
|
|
'crypto': {
|
|
'max_age': 300, # 5 minutes (crypto trades 24/7)
|
|
'memory_ttl': 600,
|
|
'force_refresh': False
|
|
},
|
|
|
|
# Sports data
|
|
'sports_recent': {
|
|
'max_age': 1800, # 30 minutes
|
|
'memory_ttl': 3600,
|
|
'force_refresh': False
|
|
},
|
|
'sports_upcoming': {
|
|
'max_age': 10800, # 3 hours
|
|
'memory_ttl': 21600,
|
|
'force_refresh': False
|
|
},
|
|
'sports_schedules': {
|
|
'max_age': 86400, # 24 hours
|
|
'memory_ttl': 172800,
|
|
'force_refresh': False
|
|
},
|
|
'leaderboard': {
|
|
'max_age': 604800, # 7 days (1 week) - football rankings updated weekly
|
|
'memory_ttl': 1209600, # 14 days in memory
|
|
'force_refresh': False
|
|
},
|
|
|
|
# News and odds
|
|
'news': {
|
|
'max_age': 3600, # 1 hour
|
|
'memory_ttl': 7200,
|
|
'force_refresh': False
|
|
},
|
|
'odds': {
|
|
'max_age': 1800, # 30 minutes for upcoming games
|
|
'memory_ttl': 3600,
|
|
'force_refresh': False
|
|
},
|
|
'odds_live': {
|
|
'max_age': 120, # 2 minutes for live games (odds change rapidly)
|
|
'memory_ttl': 240,
|
|
'force_refresh': False
|
|
},
|
|
|
|
# Static/stable data
|
|
'team_info': {
|
|
'max_age': 604800, # 1 week
|
|
'memory_ttl': 1209600,
|
|
'force_refresh': False
|
|
},
|
|
'logos': {
|
|
'max_age': 2592000, # 30 days
|
|
'memory_ttl': 5184000,
|
|
'force_refresh': False
|
|
},
|
|
|
|
# Default fallback
|
|
'default': {
|
|
'max_age': 300, # 5 minutes
|
|
'memory_ttl': 600,
|
|
'force_refresh': False
|
|
}
|
|
}
|
|
|
|
return strategies.get(data_type, strategies['default'])
|
|
|
|
def get_data_type_from_key(self, key: str) -> str:
|
|
"""
|
|
Determine the appropriate cache strategy based on the cache key.
|
|
This helps automatically select the right cache duration.
|
|
|
|
Args:
|
|
key: Cache key
|
|
|
|
Returns:
|
|
Data type string for strategy lookup
|
|
"""
|
|
key_lower = key.lower()
|
|
|
|
# Odds data — checked before the generic 'live' block below because
|
|
# live-odds cache keys (e.g. odds_espn_basketball_nba_<id>_live) contain
|
|
# both 'odds' AND 'live'. Without this ordering the 'live' check below
|
|
# would match first and return 'sports_live' (30 s TTL) instead of the
|
|
# correct 'odds_live' (120 s TTL).
|
|
if 'odds' in key_lower:
|
|
if any(x in key_lower for x in ['live', 'current']):
|
|
return 'odds_live' # Live odds change more frequently
|
|
return 'odds' # Regular odds for upcoming games
|
|
|
|
# Live sports data
|
|
if any(x in key_lower for x in ['live', 'current', 'scoreboard']):
|
|
return 'sports_live'
|
|
|
|
# Weather data
|
|
if 'weather' in key_lower:
|
|
return 'weather_current'
|
|
|
|
# Market data
|
|
if 'stock' in key_lower or 'crypto' in key_lower:
|
|
if 'crypto' in key_lower:
|
|
return 'crypto'
|
|
return 'stocks'
|
|
|
|
# News data
|
|
if 'news' in key_lower:
|
|
return 'news'
|
|
|
|
# Sports schedules and team info
|
|
if any(x in key_lower for x in ['schedule', 'team_map', 'league']):
|
|
return 'sports_schedules'
|
|
|
|
# Recent games (last few hours)
|
|
if 'recent' in key_lower:
|
|
return 'sports_recent'
|
|
|
|
# Upcoming games
|
|
if 'upcoming' in key_lower:
|
|
return 'sports_upcoming'
|
|
|
|
# Static data like logos, team info
|
|
if any(x in key_lower for x in ['logo', 'team_info', 'config']):
|
|
return 'team_info'
|
|
|
|
# Default fallback
|
|
return 'default'
|
|
|
|
def get_sport_key_from_cache_key(self, key: str) -> Optional[str]:
|
|
"""
|
|
Extract sport key from cache key to determine appropriate live_update_interval.
|
|
|
|
Args:
|
|
key: Cache key
|
|
|
|
Returns:
|
|
Sport key or None if not found
|
|
"""
|
|
key_lower = key.lower()
|
|
|
|
# Map cache key patterns to sport keys
|
|
sport_patterns = {
|
|
'nfl': ['nfl'],
|
|
'nba': ['nba', 'basketball'],
|
|
'mlb': ['mlb', 'baseball'],
|
|
'nhl': ['nhl', 'hockey'],
|
|
'soccer': ['soccer'],
|
|
'ncaa_fb': ['ncaa_fb', 'ncaafb', 'college_football'],
|
|
'ncaa_baseball': ['ncaa_baseball', 'college_baseball'],
|
|
'ncaam_basketball': ['ncaam_basketball', 'college_basketball'],
|
|
'milb': ['milb', 'minor_league'],
|
|
}
|
|
|
|
for sport_key, patterns in sport_patterns.items():
|
|
if any(pattern in key_lower for pattern in patterns):
|
|
return sport_key
|
|
|
|
return None
|
|
|
|
def is_market_open(self) -> bool:
|
|
"""
|
|
Check if the US stock market is currently open.
|
|
|
|
Returns:
|
|
True if market is open, False otherwise
|
|
"""
|
|
et_tz = pytz.timezone('America/New_York')
|
|
now = datetime.now(et_tz)
|
|
|
|
# Check if it's a weekday
|
|
if now.weekday() >= 5: # 5 = Saturday, 6 = Sunday
|
|
return False
|
|
|
|
# Convert current time to ET
|
|
current_time = now.time()
|
|
market_open = datetime.strptime('09:30', '%H:%M').time()
|
|
market_close = datetime.strptime('16:00', '%H:%M').time()
|
|
|
|
return market_open <= current_time <= market_close
|
|
|