mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-04-10 13:02:59 +00:00
remove duplicate display duration from Odds ticker, add milb and mlb recent game debug messages, some cache changes that may prove to be a mistake
This commit is contained in:
Chuck
173
CACHE_STRATEGY.md
Normal file
173
CACHE_STRATEGY.md
Normal file
@@ -0,0 +1,173 @@
|
||||
# LEDMatrix Cache Strategy Analysis
|
||||
|
||||
## Current Implementation
|
||||
|
||||
Your LEDMatrix system uses a sophisticated multi-tier caching strategy that balances data freshness with API efficiency.
|
||||
|
||||
### Cache Duration Categories
|
||||
|
||||
#### 1. **Ultra Time-Sensitive Data (15-60 seconds)**
|
||||
- **Live Sports Scores**: Now respects sport-specific `live_update_interval` configuration
|
||||
- Soccer live data: Uses `soccer_scoreboard.live_update_interval` (default: 60 seconds)
|
||||
- NFL live data: Uses `nfl_scoreboard.live_update_interval` (default: 60 seconds)
|
||||
- NHL live data: Uses `nhl_scoreboard.live_update_interval` (default: 60 seconds)
|
||||
- NBA live data: Uses `nba_scoreboard.live_update_interval` (default: 60 seconds)
|
||||
- MLB live data: Uses `mlb.live_update_interval` (default: 60 seconds)
|
||||
- NCAA sports: Use respective `live_update_interval` configurations (default: 60 seconds)
|
||||
- **Current Weather**: 5 minutes (300 seconds)
|
||||
|
||||
#### 2. **Market Data (5-10 minutes)**
|
||||
- **Stocks**: 10 minutes (600 seconds) - market hours aware
|
||||
- **Crypto**: 5 minutes (300 seconds) - 24/7 trading
|
||||
- **Stock News**: 1 hour (3600 seconds)
|
||||
|
||||
#### 3. **Sports Data (5 minutes to 24 hours)**
|
||||
- **Recent Games**: 5 minutes (300 seconds)
|
||||
- **Upcoming Games**: 1 hour (3600 seconds)
|
||||
- **Season Schedules**: 24 hours (86400 seconds)
|
||||
- **Team Information**: 1 week (604800 seconds)
|
||||
|
||||
#### 4. **Static Data (1 week to 30 days)**
|
||||
- **Team Logos**: 30 days (2592000 seconds)
|
||||
- **Configuration Data**: 1 week (604800 seconds)
|
||||
|
||||
### Smart Cache Invalidation
|
||||
|
||||
Beyond time limits, the system uses content-based invalidation:
|
||||
|
||||
```python
|
||||
def has_data_changed(self, data_type: str, new_data: Dict[str, Any]) -> bool:
|
||||
"""Check if data has changed from cached version."""
|
||||
```
|
||||
|
||||
- **Weather**: Compares temperature and conditions
|
||||
- **Stocks**: Compares prices (only during market hours)
|
||||
- **Sports**: Compares scores, game status, inning details
|
||||
- **News**: Compares headlines and article IDs
|
||||
|
||||
### Market-Aware Caching
|
||||
|
||||
For stocks, the system extends cache duration during off-hours:
|
||||
|
||||
```python
|
||||
def _is_market_open(self) -> bool:
|
||||
"""Check if the US stock market is currently open."""
|
||||
# Only invalidates cache during market hours
|
||||
```
|
||||
|
||||
## Enhanced Cache Strategy
|
||||
|
||||
### Sport-Specific Live Update Intervals
|
||||
|
||||
The cache manager now automatically respects the `live_update_interval` configuration for each sport:
|
||||
|
||||
```python
|
||||
def get_sport_live_interval(self, sport_key: str) -> int:
|
||||
"""Get the live_update_interval for a specific sport from config."""
|
||||
config = self.config_manager.get_config()
|
||||
sport_config = config.get(f"{sport_key}_scoreboard", {})
|
||||
return sport_config.get("live_update_interval", 30)
|
||||
```
|
||||
|
||||
### Automatic Sport Detection
|
||||
|
||||
The cache manager automatically detects the sport from cache keys:
|
||||
|
||||
```python
|
||||
def get_sport_key_from_cache_key(self, key: str) -> Optional[str]:
|
||||
"""Extract sport key from cache key to determine appropriate live_update_interval."""
|
||||
# Maps cache key patterns to sport keys
|
||||
sport_patterns = {
|
||||
'nfl': ['nfl', 'football'],
|
||||
'nba': ['nba', 'basketball'],
|
||||
'mlb': ['mlb', 'baseball'],
|
||||
'nhl': ['nhl', 'hockey'],
|
||||
'soccer': ['soccer', 'football'],
|
||||
# ... etc
|
||||
}
|
||||
```
|
||||
|
||||
### Configuration Examples
|
||||
|
||||
**Current Configuration (config/config.json):**
|
||||
```json
|
||||
{
|
||||
"nfl_scoreboard": {
|
||||
"live_update_interval": 30,
|
||||
"enabled": true
|
||||
},
|
||||
"soccer_scoreboard": {
|
||||
"live_update_interval": 30,
|
||||
"enabled": false
|
||||
},
|
||||
"mlb": {
|
||||
"live_update_interval": 30,
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Cache Behavior:**
|
||||
- NFL live data: 30-second cache (from config)
|
||||
- Soccer live data: 30-second cache (from config)
|
||||
- MLB live data: 30-second cache (from config)
|
||||
|
||||
### Fallback Strategy
|
||||
|
||||
If configuration is unavailable, the system uses sport-specific defaults:
|
||||
|
||||
```python
|
||||
default_intervals = {
|
||||
'soccer': 60, # Soccer default
|
||||
'nfl': 60, # NFL default
|
||||
'nhl': 60, # NHL default
|
||||
'nba': 60, # NBA default
|
||||
'mlb': 60, # MLB default
|
||||
'milb': 60, # Minor league default
|
||||
'ncaa_fb': 60, # College football default
|
||||
'ncaa_baseball': 60, # College baseball default
|
||||
'ncaam_basketball': 60, # College basketball default
|
||||
}
|
||||
```
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Automatic Sport Detection
|
||||
```python
|
||||
# Cache manager automatically detects NFL and uses nfl_scoreboard.live_update_interval
|
||||
cached_data = cache_manager.get_with_auto_strategy("nfl_live_20241201")
|
||||
|
||||
# Cache manager automatically detects soccer and uses soccer_scoreboard.live_update_interval
|
||||
cached_data = cache_manager.get_with_auto_strategy("soccer_live_20241201")
|
||||
```
|
||||
|
||||
### Manual Sport Specification
|
||||
```python
|
||||
# Explicitly specify sport for custom cache keys
|
||||
cached_data = cache_manager.get_cached_data_with_strategy("custom_live_key", "sports_live")
|
||||
```
|
||||
|
||||
## Benefits
|
||||
|
||||
1. **Configuration-Driven**: Cache respects your sport-specific settings
|
||||
2. **Automatic Detection**: No manual cache duration management needed
|
||||
3. **Sport-Optimized**: Each sport uses its appropriate update interval
|
||||
4. **Backward Compatible**: Existing code continues to work
|
||||
5. **Flexible**: Easy to adjust intervals per sport in config
|
||||
|
||||
## Migration
|
||||
|
||||
The enhanced cache manager is backward compatible. Existing code will automatically benefit from sport-specific intervals without any changes needed.
|
||||
|
||||
To customize intervals for specific sports, simply update the `live_update_interval` in your `config/config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"nfl_scoreboard": {
|
||||
"live_update_interval": 15 // More aggressive for NFL
|
||||
},
|
||||
"mlb": {
|
||||
"live_update_interval": 45 // Slower pace for MLB
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -38,7 +38,7 @@
|
||||
"hourly_forecast": 15,
|
||||
"daily_forecast": 15,
|
||||
"stock_news": 20,
|
||||
"odds_ticker": 45,
|
||||
"odds_ticker": 60,
|
||||
"nhl_live": 30,
|
||||
"nhl_recent": 20,
|
||||
"nhl_upcoming": 20,
|
||||
@@ -118,7 +118,6 @@
|
||||
"update_interval": 3600,
|
||||
"scroll_speed": 1,
|
||||
"scroll_delay": 0.01,
|
||||
"display_duration": 60,
|
||||
"loop": true
|
||||
},
|
||||
"calendar": {
|
||||
|
||||
@@ -36,6 +36,14 @@ class CacheManager:
|
||||
self._memory_cache_timestamps = {}
|
||||
self._cache_lock = threading.Lock()
|
||||
|
||||
# Initialize config manager for sport-specific intervals
|
||||
try:
|
||||
from src.config_manager import ConfigManager
|
||||
self.config_manager = ConfigManager()
|
||||
except ImportError:
|
||||
self.config_manager = None
|
||||
self.logger.warning("ConfigManager not available, using default cache intervals")
|
||||
|
||||
def _get_writable_cache_dir(self) -> Optional[str]:
|
||||
"""Tries to find or create a writable cache directory in a few common locations."""
|
||||
# Attempt 1: User's home directory (handling sudo)
|
||||
@@ -400,4 +408,227 @@ class CacheManager:
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Failed to set up persistent cache directory: {e}")
|
||||
return False
|
||||
return False
|
||||
|
||||
def get_sport_live_interval(self, sport_key: str) -> int:
|
||||
"""
|
||||
Get the live_update_interval for a specific sport from config.
|
||||
Falls back to default values if config is not available.
|
||||
"""
|
||||
if not self.config_manager:
|
||||
# Default intervals - all sports use 60 seconds as default
|
||||
default_intervals = {
|
||||
'soccer': 60, # Soccer default
|
||||
'nfl': 60, # NFL default
|
||||
'nhl': 60, # NHL default
|
||||
'nba': 60, # NBA default
|
||||
'mlb': 60, # MLB default
|
||||
'milb': 60, # Minor league default
|
||||
'ncaa_fb': 60, # College football default
|
||||
'ncaa_baseball': 60, # College baseball default
|
||||
'ncaam_basketball': 60, # College basketball default
|
||||
}
|
||||
return default_intervals.get(sport_key, 60)
|
||||
|
||||
try:
|
||||
config = self.config_manager.get_config()
|
||||
sport_config = config.get(f"{sport_key}_scoreboard", {})
|
||||
return sport_config.get("live_update_interval", 60) # Default to 60 seconds
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Could not get live_update_interval for {sport_key}: {e}")
|
||||
return 60 # Default to 60 seconds
|
||||
|
||||
def get_cache_strategy(self, data_type: str, sport_key: str = None) -> Dict[str, Any]:
|
||||
"""
|
||||
Get cache strategy for different data types.
|
||||
Now respects sport-specific live_update_interval configurations.
|
||||
"""
|
||||
# Get sport-specific live interval if provided
|
||||
live_interval = None
|
||||
if sport_key and data_type in ['sports_live', 'live_scores']:
|
||||
live_interval = self.get_sport_live_interval(sport_key)
|
||||
|
||||
strategies = {
|
||||
# Ultra time-sensitive data (live scores, current weather)
|
||||
'live_scores': {
|
||||
'max_age': live_interval or 15, # Use sport-specific interval
|
||||
'memory_ttl': (live_interval or 15) * 2, # 2x for memory cache
|
||||
'force_refresh': True
|
||||
},
|
||||
'sports_live': {
|
||||
'max_age': live_interval or 30, # Use sport-specific interval
|
||||
'memory_ttl': (live_interval or 30) * 2,
|
||||
'force_refresh': True
|
||||
},
|
||||
'weather_current': {
|
||||
'max_age': 300, # 5 minutes
|
||||
'memory_ttl': 600,
|
||||
'force_refresh': False
|
||||
},
|
||||
|
||||
# Market data (stocks, crypto)
|
||||
'stocks': {
|
||||
'max_age': 600, # 10 minutes
|
||||
'memory_ttl': 1200,
|
||||
'market_hours_only': True,
|
||||
'force_refresh': False
|
||||
},
|
||||
'crypto': {
|
||||
'max_age': 300, # 5 minutes (crypto trades 24/7)
|
||||
'memory_ttl': 600,
|
||||
'force_refresh': False
|
||||
},
|
||||
|
||||
# Sports data
|
||||
'sports_recent': {
|
||||
'max_age': 300, # 5 minutes
|
||||
'memory_ttl': 600,
|
||||
'force_refresh': False
|
||||
},
|
||||
'sports_upcoming': {
|
||||
'max_age': 3600, # 1 hour
|
||||
'memory_ttl': 7200,
|
||||
'force_refresh': False
|
||||
},
|
||||
'sports_schedules': {
|
||||
'max_age': 86400, # 24 hours
|
||||
'memory_ttl': 172800,
|
||||
'force_refresh': False
|
||||
},
|
||||
|
||||
# News and odds
|
||||
'news': {
|
||||
'max_age': 3600, # 1 hour
|
||||
'memory_ttl': 7200,
|
||||
'force_refresh': False
|
||||
},
|
||||
'odds': {
|
||||
'max_age': 3600, # 1 hour
|
||||
'memory_ttl': 7200,
|
||||
'force_refresh': False
|
||||
},
|
||||
|
||||
# Static/stable data
|
||||
'team_info': {
|
||||
'max_age': 604800, # 1 week
|
||||
'memory_ttl': 1209600,
|
||||
'force_refresh': False
|
||||
},
|
||||
'logos': {
|
||||
'max_age': 2592000, # 30 days
|
||||
'memory_ttl': 5184000,
|
||||
'force_refresh': False
|
||||
},
|
||||
|
||||
# Default fallback
|
||||
'default': {
|
||||
'max_age': 300, # 5 minutes
|
||||
'memory_ttl': 600,
|
||||
'force_refresh': False
|
||||
}
|
||||
}
|
||||
|
||||
return strategies.get(data_type, strategies['default'])
|
||||
|
||||
def get_data_type_from_key(self, key: str) -> str:
|
||||
"""
|
||||
Determine the appropriate cache strategy based on the cache key.
|
||||
This helps automatically select the right cache duration.
|
||||
"""
|
||||
key_lower = key.lower()
|
||||
|
||||
# Live sports data
|
||||
if any(x in key_lower for x in ['live', 'current', 'scoreboard']):
|
||||
if 'soccer' in key_lower:
|
||||
return 'sports_live' # Soccer live data is very time-sensitive
|
||||
return 'sports_live'
|
||||
|
||||
# Weather data
|
||||
if 'weather' in key_lower:
|
||||
return 'weather_current'
|
||||
|
||||
# Market data
|
||||
if 'stock' in key_lower or 'crypto' in key_lower:
|
||||
if 'crypto' in key_lower:
|
||||
return 'crypto'
|
||||
return 'stocks'
|
||||
|
||||
# News data
|
||||
if 'news' in key_lower:
|
||||
return 'news'
|
||||
|
||||
# Odds data
|
||||
if 'odds' in key_lower:
|
||||
return 'odds'
|
||||
|
||||
# Sports schedules and team info
|
||||
if any(x in key_lower for x in ['schedule', 'team_map', 'league']):
|
||||
return 'sports_schedules'
|
||||
|
||||
# Recent games (last few hours)
|
||||
if 'recent' in key_lower:
|
||||
return 'sports_recent'
|
||||
|
||||
# Upcoming games
|
||||
if 'upcoming' in key_lower:
|
||||
return 'sports_upcoming'
|
||||
|
||||
# Static data like logos, team info
|
||||
if any(x in key_lower for x in ['logo', 'team_info', 'config']):
|
||||
return 'team_info'
|
||||
|
||||
# Default fallback
|
||||
return 'default'
|
||||
|
||||
def get_sport_key_from_cache_key(self, key: str) -> Optional[str]:
|
||||
"""
|
||||
Extract sport key from cache key to determine appropriate live_update_interval.
|
||||
"""
|
||||
key_lower = key.lower()
|
||||
|
||||
# Map cache key patterns to sport keys
|
||||
sport_patterns = {
|
||||
'nfl': ['nfl', 'football'],
|
||||
'nba': ['nba', 'basketball'],
|
||||
'mlb': ['mlb', 'baseball'],
|
||||
'nhl': ['nhl', 'hockey'],
|
||||
'soccer': ['soccer', 'football'],
|
||||
'ncaa_fb': ['ncaa_fb', 'ncaafb', 'college_football'],
|
||||
'ncaa_baseball': ['ncaa_baseball', 'college_baseball'],
|
||||
'ncaam_basketball': ['ncaam_basketball', 'college_basketball'],
|
||||
'milb': ['milb', 'minor_league'],
|
||||
}
|
||||
|
||||
for sport_key, patterns in sport_patterns.items():
|
||||
if any(pattern in key_lower for pattern in patterns):
|
||||
return sport_key
|
||||
|
||||
return None
|
||||
|
||||
def get_cached_data_with_strategy(self, key: str, data_type: str = 'default') -> Optional[Dict]:
|
||||
"""
|
||||
Get data from cache using data-type-specific strategy.
|
||||
Now respects sport-specific live_update_interval configurations.
|
||||
"""
|
||||
# Extract sport key for live sports data
|
||||
sport_key = None
|
||||
if data_type in ['sports_live', 'live_scores']:
|
||||
sport_key = self.get_sport_key_from_cache_key(key)
|
||||
|
||||
strategy = self.get_cache_strategy(data_type, sport_key)
|
||||
max_age = strategy['max_age']
|
||||
|
||||
# For market data, check if market is open
|
||||
if strategy.get('market_hours_only', False) and not self._is_market_open():
|
||||
# During off-hours, extend cache duration
|
||||
max_age *= 4 # 4x longer cache during off-hours
|
||||
|
||||
return self.get_cached_data(key, max_age)
|
||||
|
||||
def get_with_auto_strategy(self, key: str) -> Optional[Dict]:
|
||||
"""
|
||||
Get cached data using automatically determined strategy.
|
||||
Now respects sport-specific live_update_interval configurations.
|
||||
"""
|
||||
data_type = self.get_data_type_from_key(key)
|
||||
return self.get_cached_data_with_strategy(key, data_type)
|
||||
Reference in New Issue
Block a user