feat(fetch): shared fetch service, stage 1 (pooling, merging, host budgets, counters) (#702)

Core's own HTTP fetch paths (APIHelper, fetch_espn_scoreboard and its date chunks, BackgroundDataService, BaseOddsManager.get_odds) go through one service in src/common/fetch_service.py: shared connection pools per retry policy, merged identical in-flight GETs, per-host token-bucket budgets (fetch_service.rate_limits), and per-plugin request counters published to GET /api/v3/plugins/fetch-stats. Return values, exceptions, cache keys, TTLs and retry policies are unchanged. Core-internal in this release; plugins should not import it directly yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-10-01 15:38:11 -04:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 9edeb6da14
commit 4be53d048b
19 changed files with 2407 additions and 31 deletions
@@ -1,4 +1,4 @@
"""Plugin health, resource metrics and resource limits.
"""Plugin health, resource metrics, fetch statistics and resource limits.
The display process records health and metrics to the shared on-disk cache;
these routes read (and reset) that published state through a tracker and a
@@ -193,6 +193,24 @@ def reset_plugin_metrics(plugin_id):
})
@api_v3.route('/plugins/fetch-stats', methods=['GET'])
def get_fetch_stats():
"""Network requests per plugin and per host, as the display counts them.
Read-only. The display's fetch service (src/common/fetch_service.py)
publishes its counters to the shared cache at most once a minute when
they change; this returns that snapshot judged for staleness:
``data.status`` is ``live``, ``stale``, ``stopped`` or ``unknown``, and
``data.data`` the snapshot (None when unknown). Counters are cumulative
since the display started.
"""
from src.common.fetch_service import read_fetch_stats
return jsonify({
'status': 'success',
'data': read_fetch_stats(getattr(api_v3, 'cache_manager', None)),
})
@api_v3.route('/plugins/limits/<plugin_id>', methods=['GET', 'POST'])
def manage_plugin_limits(plugin_id):
"""Get or set resource limits for a plugin.