perf(espn): remember settled day chunks between window refreshes

Since ESPN started rejecting date ranges, every scoreboard's hourly
Recent/Upcoming refresh re-asks its 22-day window (14 back, 7 ahead) one
day at a time. Measured on hdpi 2026-10-02 (NFL, college football, MLB,
college baseball, NHL): the hourly refresh was ~270 of 321 ESPN requests
and ~21 of 24.6MB in the hour. Days that ended three or more days ago
cannot change, and they were 68% of the window's bytes (6.9 of 10.2MB).

_fetch_one_chunk now keeps a settled chunk (last day <= UTC today - 3) in
memory for 24h as zlib-compressed JSON, keyed by URL, the other params
and the chunk, and answers it from there. Both range paths go through it:
fetch_espn_scoreboard and BackgroundDataService._fetch_in_date_chunks.
Each hit is parsed afresh, failed and capped chunks are not stored, and
the memory is bounded (512 entries / 8MB compressed). Against live ESPN,
a second refresh of the five hdpi windows went from 111 requests and
10.18MB to 50 requests and 3.23MB with identical events; the memory held
60 entries in 585KB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
ChuckBuilds
2026-10-03 22:24:59 -04:00
co-authored by Claude Opus 5.5
parent 07abd87d5e
commit 3ea1fd42df
3 changed files with 263 additions and 2 deletions
+18
View File
@@ -343,6 +343,24 @@ def _hermetic_unit_refresh(monkeypatch, tmp_path_factory):
monkeypatch.setattr(unit_refresh, 'SYSTEMD_DIR', str(tmp_path_factory.getbasetemp() / 'no-systemd'))
@pytest.fixture(autouse=True)
def _forget_settled_espn_chunks(monkeypatch):
"""src.common.espn_dates remembers past days process-wide; tests fake
different answers for the same dates, so none may inherit another's.
"Today" is also pinned to 2000-01-01, so no date a test uses counts as
settled unless the test says so (by pinning _utc_today itself). Without
that, a test asking for last month twice passes while that month is
recent and fails once it is three days old: the second ask is answered
from memory."""
from datetime import date
from src.common import espn_dates
monkeypatch.setattr(espn_dates, "_utc_today", lambda: date(2000, 1, 1))
espn_dates.clear_settled_chunk_cache()
yield
espn_dates.clear_settled_chunk_cache()
@pytest.fixture(autouse=True)
def reset_logging():
"""Reset logging configuration before each test."""