fix(espn): fetch a window's edge months whole, cap chunk requests per process

A soccer board with eight leagues logged ~90 NameResolutionErrors for
site.api.espn.com within a minute of every start on ledpi, plus
"update() timed out". ESPN rejects date ranges, so every league's
fortnight-either-side window became 29 day requests, several managers
fetched it at once, and each window got its own six chunk threads:
~450 requests per start with ~45 in flight, each request past a
session's pool a new connection and a new DNS lookup.

- fetch_espn_date_chunks() asks for a partial edge month whole once the
  window covers ESPN_MONTH_COVER_MIN_DAYS (7) of its days, and trims the
  answer to the window by each event's US Eastern start date, the day
  ESPN's dates=YYYYMMDD means (417/417 live soccer events matched). A
  29-day window over two months is 2 requests, not 29. A live poll's
  1-2 days stay day requests. A capped trimmed month re-asks only the
  window's days; an undatable event is kept. New espn_request_chunks().
- Chunk requests share one process-wide cap of ESPN_CHUNK_WORKERS in
  flight instead of six per window.
- A new process starts as if a range had just been rejected, so it no
  longer spends a doomed 400 per window at every start.

Live ESPN, soccer-scoreboard 2.39.2, alternating runs: ~450 -> 46
requests per start, peak ~45 -> 13 in flight, ~75 -> 30 DNS lookups.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-10-03 22:35:09 -04:00
co-authored by Claude Opus 5.5
parent 41192b9588
commit 8da7fc01d7
5 changed files with 384 additions and 41 deletions
+2 -2
View File
@@ -670,14 +670,14 @@ class TestCallerIdentity:
assert _counters(global_service, plugin="football-scoreboard")["requests"] == 1
def test_espn_chunks_on_worker_threads_count_against_the_caller(self, global_service):
from src.common.espn_dates import espn_date_chunks, fetch_espn_date_chunks, parse_espn_date_range
from src.common.espn_dates import espn_request_chunks, fetch_espn_date_chunks, parse_espn_date_range
session = FakeSession(lambda url, kw: make_response(body=b'{"events": []}', url=url))
dates = "20260801-20261015"
with plugin_scope("baseball-scoreboard"):
fetch_espn_date_chunks(session, "https://site.api.espn.com/s/scoreboard",
params={"dates": dates})
chunks = len(espn_date_chunks(*parse_espn_date_range(dates)))
chunks = len(espn_request_chunks(*parse_espn_date_range(dates)))
assert chunks > 1
assert len(session.calls) == chunks
assert _counters(global_service, plugin="baseball-scoreboard")["requests"] == chunks