perf(fetch): cache ESPN scoreboard windows without the parts nothing reads (#749)

The sports scoreboards cache their Recent/Upcoming window as the raw ESPN
response, and it stays parsed in the memory tier while fresh. Measured on
hdpi, most of it is never drawn: per-team stat leaders, athlete cards
(featuredAthletes, probables), team and event links, headlines, video
highlights and geo broadcasts. None of those keys is read by core or by any
plugin in ledmatrix-plugins.

BackgroundDataService now drops them from an ESPN /scoreboard response
before caching and delivering it (src/common/espn_payload.py), keeping
everything else. On the five hdpi windows that is 10.6MB -> 3.0MB of JSON
and ~40MB -> ~12MB of parsed objects, and parsing an expired window gets
3-4x cheaper. submit_fetch_request(slim_payload=False) caches a response
whole.


Claude-Session: https://claude.ai/code/session_01BkfgXMqqwn2w4NN7LRzhxy

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-10-06 08:50:37 -04:00
committed by GitHub
co-authored by Claude Opus 5.5
parent ec117a35a1
commit 7c5fa9cfdb
3 changed files with 279 additions and 1 deletions
+20 -1
View File
@@ -34,6 +34,7 @@ from src.common.fetch_service import (
plugin_scope,
share_connection_pool,
)
from src.common.espn_payload import is_espn_scoreboard_url, slim_scoreboard_payload
from src.common.espn_dates import (
RANGE_RETRY_SECONDS,
_note_range_rejected,
@@ -83,6 +84,10 @@ class FetchRequest:
# the cache with the callbacks suppressed -- joiners waiting forever for a
# fetch that did, in fact, succeed.
commit_claimed: bool = False
# Trim an ESPN scoreboard response before it is cached and delivered
# (src/common/espn_payload.py). Set by whoever created the request; a
# submitter that joins the fetch gets the same payload.
slim_payload: bool = True
result: Optional[Any] = None
error: Optional[str] = None
# The plugin that submitted the request, so the fetch service counts the
@@ -249,7 +254,8 @@ class BackgroundDataService:
timeout: Optional[int] = None,
max_retries: int = 3,
priority: int = 1,
callback: Optional[Callable] = None) -> str:
callback: Optional[Callable] = None,
slim_payload: bool = True) -> str:
"""
Submit a background fetch request.
@@ -265,6 +271,11 @@ class BackgroundDataService:
priority: Accepted for compatibility and ignored; requests run in
submission order.
callback: Optional callback function when request completes
slim_payload: Drop the parts of an ESPN scoreboard response no
scoreboard reads (stat leaders, athlete cards, links,
headlines, highlights) before caching it; see
src/common/espn_payload.py. Only ESPN /scoreboard URLs are
touched. Pass False to cache the response whole.
Returns:
Request ID for tracking the fetch operation
@@ -336,6 +347,7 @@ class BackgroundDataService:
priority=priority,
callback=callback,
owner=owner,
slim_payload=slim_payload,
)
with self._lock:
@@ -497,6 +509,13 @@ class BackgroundDataService:
)
return result
# Most of an ESPN scoreboard response is never drawn, and the
# cached copy stays parsed in the memory tier while it is fresh.
# Trimmed before the write so the cache, request.result and the
# callbacks all see the same payload. See src/common/espn_payload.py.
if request.slim_payload and is_espn_scoreboard_url(request.url):
slim_scoreboard_payload(data)
# Cache the data
self.cache_manager.set(request.cache_key, data)