fix(background): release the payload after every callback, not after each one (#509)

Callers that join an in-flight fetch share one FetchResult. #499 released the
payload inside the delivery loop, so the first callback got the data and every
joiner got `result.data is None`.

That is not a quiet degradation. Consumers read `result.data.get('events')`, so
they raise AttributeError -- which the delivery loop catches and logs. The
entire failure surfaced as one line:

    ERROR - src.background_data_service - Error in callback for request
    nhl_2026_...: 'NoneType' object has no attribute 'get'

and a manager that silently never received its schedule. Seen on hardware:
NHLRecentManager logs "Background fetch completed for 2026: 1000 events" and
the very next line is the error, from NHLUpcomingManager's callback on the same
request -- which had already logged "No events found in shared data."

Deduplication is the normal case, not a corner. A sport's recent, upcoming and
live managers all want the same season schedule, so the second and third are
joiners on almost every cycle. _release_payload's own docstring said "once A
callback has been handed the data", singular, which is the assumption that
broke: the loop above it was written for many, and says so.

Moved after the loop, and guarded on `callbacks` being non-empty. The guard
matters: a request submitted without a callback must keep its payload, because
polling get_result() is then the only way to collect it. The per-delivery
release got that right by accident -- an empty list never entered the loop body
-- and the existing test for it caught the omission.

test_background_payload_release.py gains TestJoinersAllGetTheData: two
submitters on one in-flight cache_key, asserting both are handed a populated
payload, plus that the memory fix still happens once they have all had it.
test_background_fetch_dedupe.py already proved the joiner's callback FIRES; it
never checked what the callback received, which is the gap that let this
through.

Verified the new test bites: restoring the release inside the loop fails it
with "'second' was handed a released payload".

Full suite 3716 passed, 6 skipped.


Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-01 09:54:04 -04:00
committed by GitHub
co-authored by Claude Opus 5
parent 9b522d412c
commit 154525beb8
2 changed files with 114 additions and 4 deletions
+21 -4
View File
@@ -478,8 +478,22 @@ class BackgroundDataService:
cb(result)
except Exception as e:
logger.error(f"Error in callback for request {request.id}: {e}")
# Delivered. Drop both references -- they point at the same
# object, so one survivor keeps the whole payload resident.
# Released AFTER the loop, not inside it. Every callback here holds
# the same FetchResult, so releasing per-delivery handed the first
# one the data and every joiner `result.data is None` -- which is
# not a quiet degradation: they read `result.data.get('events')` and
# raise AttributeError, which this very loop catches and logs, so
# the symptom was one ERROR line and a manager that silently never
# got its schedule. Deduplication is the normal case, not a corner:
# a sport's recent, upcoming and live managers all ride one season
# fetch.
#
# Guarded on `callbacks`, because a request submitted without one
# has no other way to collect its payload than polling get_result().
# The old per-delivery release got that right by accident: an empty
# list never entered the loop body.
if callbacks:
self._release_payload(result)
request.result = None
@@ -489,8 +503,11 @@ class BackgroundDataService:
def _release_payload(result: FetchResult) -> None:
"""Drop a delivered payload, keeping the result's status and timings.
Only called once a callback has been handed the data. Consumers read
fetched data back from the cache under ``cache_key``; the copy carried
Only called once EVERY callback has been handed the data -- callers
that joined an in-flight fetch share this object, so releasing between
deliveries strips the payload out from under the ones still queued.
Consumers read fetched data back from the cache under ``cache_key``;
the copy carried
here was pinning a parsed season schedule -- 946 games for NCAA
football, roughly a tenth of total RAM on a 1GB Pi -- in memory until
the hourly sweep.