rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-04 14:25:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(cache): remove the cache layer's duplicate cleanup and dead lookups (#613)

Browse Source 
* refactor(cache): collapse CacheStrategy's all-60 defaults table and twin soccer branch

get_sport_live_interval() without a config manager looked the sport up in
a table where every value was 60, with 60 as the fallback; it now returns
60. get_data_type_from_key() had an `if 'soccer'` branch returning the
same 'sports_live' as its else.

test_cache_strategy_intervals pins the returned strategy for every data
type x sport key x config-manager shape; it passes unchanged on the old
code. A 2,544-entry dump of every CacheStrategy method over a wider grid
is identical before and after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(cache): drop CacheStrategy's `<sport>_scoreboard` config lookup

get_sport_live_interval() and get_cache_strategy() read live/recent/
upcoming intervals from config[f"{sport}_scoreboard"]. Those sections
belonged to the built-in scoreboards the plugin system replaced; plugin
config is keyed by plugin id ("football-scoreboard"), so on a current
config the lookup always fell through to the defaults (60 live, 1800
recent, 10800 upcoming), which are now returned directly.

The one input where this differs: a config.json upgraded from the
pre-plugin era that still carries e.g. an "nfl_scoreboard" section (no
code removes them), queried with an explicit sport key. No caller in core
or the plugin monorepo passes a sport key here -- get_with_auto_strategy
only derives one for keys classed sports_live/live_scores, and its callers
(odds managers, odds-ticker) use odds keys -- so the stale section was
unreachable in practice. A dump of every CacheStrategy method over 2,544
inputs differs from the previous commit only in those 45 legacy-config
entries; the test grid now includes that shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* perf(cache): list cache files without holding the memory-tier lock

CacheManager.list_cache_files() held the in-memory cache's lock while it
listed and stat'd the whole cache directory -- 8,864 files on a real rig
-- so every get()/set() from the display loop and plugins waited out the
scan. The lock never protected the disk: DiskCache writes and deletes
under their own lock, and a file vanishing between listdir and stat was
already handled (logged and skipped). The body is unchanged apart from
the dedent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(cache): delegate memory-tier cleanup and stats to MemoryCache

CacheManager._cleanup_memory_cache() was a line-for-line copy of
MemoryCache.cleanup(), and get_memory_cache_stats() a copy of
MemoryCache.get_stats(), both reaching into the component's private
_cache/_timestamps/_lock through "backward compatibility" aliases bound
in __init__. So the component's own cleanup and stats only ever ran in
tests, and the aliases went stale whenever the component was swapped
(test_cache_ttl_honoured does). Both now delegate, and the aliases are
gone: nothing in core, the tests, or the ledmatrix-plugins monorepo reads
them.

Behaviour is the same. Compared line by line, the two cleanups differ
only in the sort key's fallback (0 vs 0.0, which orders identically),
range+bounds check vs slice for the eviction, and the logger name on the
DEBUG summary line (src.cache_manager -> src.cache.memory_cache). A
differential run over 20,000 random memory states (str/None/garbage/
future timestamps, orphan keys, sizes 0-12, forced and throttled runs)
gives identical removed counts, resulting dicts and last-cleanup times;
the same harness catches each of three seeded mutations of
MemoryCache.cleanup. The throttle clock also moves with it:
CacheManager kept its own copy of last-cleanup, the component's is used
now, and they started equal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(background): inline the sport cache key and drop the unused request queue

get_sport_cache_key() constructed a whole CacheManager -- ConfigManager,
config parse, cache-dir probing with test-file writes -- to return
f"{sport}_{date}". It now builds the key itself in the same format as
CacheManager.generate_sport_cache_key() (UTC date, %Y%m%d); tests check
the two agree for explicit dates and, with a frozen clock at 03:30 UTC,
for the default date. Median per call on Windows: ~0.6 ms -> ~2 us
(alternating runs); on a Pi the old path also wrote a probe file per call.

request_queue was a PriorityQueue nothing ever put into: requests go
straight to the executor, so `priority` never did anything. The queue is
gone; the `priority` parameter and FetchRequest field stay (every
monorepo scoreboard passes priority=) and are documented as ignored, and
get_statistics() keeps reporting queue_size, now a literal 0 as it
always was in practice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-23 12:54:07 -04:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 269385c97c
commit 9d024f24ef
6 changed files with 416 additions and 188 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/cache/cache_strategy.py
Vendored
+23 -55
View File
@@ -1,7 +1,7 @@
"""
Cache Strategy
Manages cache strategies for different data types with sport-specific configurations.
Manages cache strategies (TTLs) for different data types.
"""
import logging
@@ -18,7 +18,8 @@ class CacheStrategy:
Initialize cache strategy manager.
Args:
config_manager: Optional ConfigManager instance for sport-specific configs
config_manager: Optional ConfigManager instance. Kept for callers
that pass one; no strategy currently reads it.
logger: Optional logger instance
"""
self.config_manager = config_manager
@@ -26,69 +27,38 @@ class CacheStrategy:
def get_sport_live_interval(self, sport_key: str) -> int:
"""
Get the live_update_interval for a specific sport from config.
Falls back to default values if config is not available.
Live-data cache interval, in seconds, for a sport: 60 for every sport.
This used to read ``live_update_interval`` from a ``<sport>_scoreboard``
config section. Those sections belonged to the built-in scoreboards
that the plugin system replaced; plugin config is keyed by plugin id
(``football-scoreboard``), so the lookup always fell back to 60.
Args:
sport_key: Sport identifier (e.g., 'nba', 'nfl')
Returns:
Live update interval in seconds
"""
if not self.config_manager:
# Default intervals - all sports use 60 seconds as default
default_intervals = {
'soccer': 60,
'nfl': 60,
'nhl': 60,
'nba': 60,
'mlb': 60,
'milb': 60,
'ncaa_fb': 60,
'ncaa_baseball': 60,
'ncaam_basketball': 60,
}
return default_intervals.get(sport_key, 60)
try:
config = self.config_manager.config
# All sports now use _scoreboard suffix
sport_config = config.get(f"{sport_key}_scoreboard", {})
return sport_config.get("live_update_interval", 60) # Default to 60 seconds
except (KeyError, AttributeError, TypeError) as e:
self.logger.warning("Could not get live_update_interval for %s: %s", sport_key, e, exc_info=True)
return 60 # Default to 60 seconds
return 60
def get_cache_strategy(self, data_type: str, sport_key: Optional[str] = None) -> Dict[str, Any]:
"""
Get cache strategy for different data types.
Now respects sport-specific live_update_interval configurations.
Args:
data_type: Type of data (e.g., 'live_scores', 'stocks', 'weather_current')
sport_key: Optional sport key for sport-specific intervals
sport_key: Optional sport key; for live data it selects the
per-sport interval from :meth:`get_sport_live_interval`
instead of the generic live default.
Returns:
Dictionary with cache strategy (max_age, memory_ttl, etc.)
"""
# Get sport-specific live interval if provided
live_interval = None
if sport_key and data_type in ['sports_live', 'live_scores']:
live_interval = self.get_sport_live_interval(sport_key)
# Try to read sport-specific config for recent/upcoming
recent_interval = None
upcoming_interval = None
if self.config_manager and sport_key:
try:
# All sports now use _scoreboard suffix
sport_cfg = self.config_manager.config.get(f"{sport_key}_scoreboard", {})
recent_interval = sport_cfg.get('recent_update_interval')
upcoming_interval = sport_cfg.get('upcoming_update_interval')
except (KeyError, AttributeError, TypeError) as e:
self.logger.debug("Could not read sport-specific recent/upcoming intervals for %s: %s",
sport_key, e, exc_info=True)
strategies = {
# Ultra time-sensitive data (live scores, current weather)
'live_scores': {
@@ -122,13 +92,13 @@ class CacheStrategy:
# Sports data
'sports_recent': {
'max_age': recent_interval or 1800, # 30 minutes default; override by config
'memory_ttl': (recent_interval or 1800) * 2,
'max_age': 1800, # 30 minutes
'memory_ttl': 3600,
'force_refresh': False
},
'sports_upcoming': {
'max_age': upcoming_interval or 10800, # 3 hours default; override by config
'memory_ttl': (upcoming_interval or 10800) * 2,
'max_age': 10800, # 3 hours
'memory_ttl': 21600,
'force_refresh': False
},
'sports_schedules': {
@@ -206,8 +176,6 @@ class CacheStrategy:
# Live sports data
if any(x in key_lower for x in ['live', 'current', 'scoreboard']):
if 'soccer' in key_lower:
return 'sports_live' # Soccer live data is very time-sensitive
return 'sports_live'
# Weather data