rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-04 14:25:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(logos): stop a failed download pinning a team to a grey box forever (#512)

Browse Source 
* fix(logos): stop a failed download pinning a team to a grey box forever

When a logo download fails, create_placeholder_logo writes a 64x64 grey PNG
under the *real* logo's filename. Every later call then hits
`if filepath.exists(): return True` and reports success, so the real logo is
never attempted again. One transient failure -- no network at boot, ESPN
blipping -- permanently costs that team its logo.

This is not hypothetical. Five of the eleven cached AFL logos in my checkout
were 384-byte stubs written in a single bad minute, and they had stayed that
way ever since; the scoreboard rendered COLL, FRE, NMFC, PORT and SYD as grey
text boxes on every card.

Placeholders are now stamped with a `ledmatrix_placeholder` PNG text chunk
carrying their creation time, and `is_placeholder_logo` recognises them. It
also matches on the placeholder's exact geometry and background colour, so the
stubs already sitting on users' disks are picked up too -- without that, this
fix would only help teams whose logos break in future. Verified against the
real stubs: all five detected, all six real logos untouched.

`download_missing_logo` now treats an existing placeholder as the failed
download it is and retries, rather than as a satisfied request. The retry is
rate-limited to PLACEHOLDER_RETRY_SECONDS (6h) so this does not trade a
permanent grey box for an ESPN request every frame; a failed retry rewrites the
placeholder, restarting the clock. The age comes from the stamp rather than
mtime, so a backup restore, an rsync, or a permissions script cannot silently
reset it.

`download_missing_logos_for_league` gets the same treatment -- a bulk pass is
exactly where a previously failed logo should get another chance -- and
`LogoHelper.load_logo_with_download` no longer accepts a stale placeholder as a
cache hit. That import is lazy and guarded so the module still works against a
core build predating the marker.

`LogoHelper._create_placeholder_logo` needs no change: it returns an in-memory
image and never writes it to disk, which is the behaviour this bug argues for.

Tests cover marked and legacy-unmarked detection, the two false-positive cases
(a real 500x500 logo, and a 64x64 image that is merely the same size), the
retry, the rate limit, and that the age survives an mtime touch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(logos): address review — unify eligibility, invalidate cache, restart back-off

Three findings from the review on #512, all confirmed against the code:

1. The three download sites each had their own idea of "already have it".
   download_missing_logos_for_league() retried *any* placeholder, ignoring the
   back-off entirely, while download_all_ncaa_football_logos() was never
   updated and still skipped placeholders forever. They now share one
   should_attempt_download(), which also covers force_download, so the sites
   cannot drift apart again. download_missing_logo() reads through the same
   helper.

2. LogoHelper.load_logo_with_download() answered from the in-memory cache
   before touching the disk, so after a stale placeholder was successfully
   replaced the *cached placeholder image* was still returned -- the real logo
   would not have appeared until the process restarted. The cache entry for
   that file (every size of it) is now dropped after a successful download.

3. A failed retry left the stale placeholder on disk with its old timestamp,
   so the next call saw it as stale again and retried immediately: a download
   attempt per call, which is precisely what the back-off exists to prevent.
   refresh_placeholder_timestamp() restamps it, and the helper calls that on
   the failure path. It refuses to touch anything that is not a placeholder.

Tests cover both bulk loops in both directions (fresh placeholder skipped,
stale one retried), the eligibility rule including force_download, the
timestamp refresh, and the two LogoHelper paths -- including that a
freshly-downloaded logo is actually what comes back rather than the cached
placeholder.

Two of the new bulk-loop tests initially passed for the wrong reason: the
fetch_teams_data stub returned {}, which is falsy, so the loops bailed before
reaching the eligibility check at all. Fixed to return a truthy payload.

Re-verified end to end: with both halves in place, rendering the AFL scoreboard
took FRE.png from a 362-byte stub to a 12,928-byte logo.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-02 13:21:07 -04:00
committed by GitHub
co-authored by Claude Opus 5
parent 6e361e05cc
commit 92f9d06af9
4 changed files with 466 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/common/logo_helper.py
+52 -2
View File
@@ -143,8 +143,10 @@ class LogoHelper:
"""
logo_path = Path(logo_path)
# Try to load existing logo first
if logo_path.exists():
# Try to load existing logo first. A placeholder written by a previous
# failed download does not count: it wears the real logo's filename, so
# trusting the file's existence is what left teams as grey boxes.
if logo_path.exists() and not self._is_stale_placeholder(logo_path):
return self.load_logo(team_abbr, logo_path, max_width, max_height)
# Download if URL provided and file doesn't exist
@@ -152,13 +154,61 @@ class LogoHelper:
try:
self.logger.info(f"Downloading logo for {team_abbr} from {logo_url}")
self._download_logo(logo_url, logo_path)
# The file on disk just changed. Any cached image for it is the
# placeholder we came here to replace, and load_logo() answers
# from the cache before touching the disk -- so without this the
# real logo would not appear until the process restarted.
self._invalidate_cached_logo(team_abbr, logo_path)
return self.load_logo(team_abbr, logo_path, max_width, max_height)
except Exception as e:
self.logger.error(f"Failed to download logo for {team_abbr}: {e}")
# The retry failed, so restart the back-off. The stale
# placeholder is still on disk with its old timestamp, and
# leaving it there means the next call retries immediately --
# a download attempt per call, which is what the back-off
# exists to prevent.
self._refresh_stale_placeholder(logo_path)
# Create placeholder if all else fails
return self._create_placeholder_logo(team_abbr, max_width, max_height)
def _invalidate_cached_logo(self, team_abbr: str, logo_path: Path) -> None:
"""Drop every cached size of one logo after its file changed on disk."""
prefix = f"{team_abbr}_{logo_path}_"
for key in [k for k in self._logo_cache if k.startswith(prefix)]:
self._logo_cache.pop(key, None)
if key in self._cache_order:
self._cache_order.remove(key)
@staticmethod
def _refresh_stale_placeholder(logo_path: Path) -> None:
"""Restart the retry back-off after a failed download attempt."""
try:
from src.logo_downloader import refresh_placeholder_timestamp
except ImportError:
return
refresh_placeholder_timestamp(logo_path)
@staticmethod
def _is_stale_placeholder(logo_path: Path) -> bool:
"""True if the file is a placeholder old enough to be worth retrying.
Imported lazily so this module keeps working against a core build whose
logo_downloader predates placeholder marking.
"""
try:
from src.logo_downloader import (
PLACEHOLDER_RETRY_SECONDS,
is_placeholder_logo,
placeholder_age_seconds,
)
except ImportError:
return False
if not is_placeholder_logo(logo_path):
return False
age = placeholder_age_seconds(logo_path)
return age is None or age >= PLACEHOLDER_RETRY_SECONDS
def get_logo_variations(self, team_abbr: str) -> List[str]:
"""
Get possible filename variations for a team abbreviation.
src/logo_downloader.py
+116 -8
View File
@@ -14,6 +14,7 @@ import json
from typing import Dict, List, Optional, Tuple
from pathlib import Path
from PIL import Image, ImageDraw, ImageFont
from PIL.PngImagePlugin import PngInfo
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from src.common.permission_utils import (
@@ -25,6 +26,96 @@ from src.common.permission_utils import (
logger = logging.getLogger(__name__)
#: PNG text key stamped into a generated placeholder so a later run can tell it
#: apart from a real logo that happens to be small.
PLACEHOLDER_MARKER = "ledmatrix_placeholder"
#: Geometry of a generated placeholder, used to recognise ones written before
#: the marker existed. Those are already on users' disks and would otherwise
#: never be retried.
PLACEHOLDER_SIZE = (64, 64)
PLACEHOLDER_BG = (100, 100, 100, 255)
#: How long a placeholder is trusted before the real logo is attempted again.
#: A placeholder means the download failed, and download failures are usually
#: transient (no network at boot, ESPN blipping). Retrying every frame would
#: hammer the API from a Pi that is also driving a panel; never retrying leaves
#: the team a grey box forever, which is the bug this exists to avoid.
PLACEHOLDER_RETRY_SECONDS = 6 * 60 * 60
def is_placeholder_logo(filepath: Path) -> bool:
"""True if the file at ``filepath`` is a generated placeholder, not a logo.
Checks the marker first, then falls back to matching the placeholder's
exact geometry and background colour so files written before the marker was
introduced are still recognised.
"""
try:
with Image.open(filepath) as img:
if img.info.get(PLACEHOLDER_MARKER):
return True
if img.size != PLACEHOLDER_SIZE:
return False
return img.convert("RGBA").getpixel((0, 0)) == PLACEHOLDER_BG
except Exception:
# Unreadable file: not provably a placeholder, and the caller's own
# error handling is better placed to deal with it.
return False
def should_attempt_download(filepath: Path, force_download: bool = False) -> bool:
"""Whether a real logo is worth (re)fetching for ``filepath``.
True when nothing is there, when the caller forced it, or when what is
there is a placeholder old enough to retry. A *fresh* placeholder says a
download just failed, so retrying it immediately would hammer the API for
a result that is very unlikely to have changed.
"""
if force_download or not filepath.exists():
return True
if not is_placeholder_logo(filepath):
return False
age = placeholder_age_seconds(filepath)
return age is None or age >= PLACEHOLDER_RETRY_SECONDS
def refresh_placeholder_timestamp(filepath: Path) -> bool:
"""Restamp a placeholder so a failed retry restarts the back-off clock.
Without this a stale placeholder stays stale: every later call sees an
expired timestamp, retries, fails, and leaves the timestamp untouched --
which is a download attempt per call, the opposite of what the back-off is
for.
"""
try:
if not is_placeholder_logo(filepath):
return False
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time()))
with Image.open(filepath) as img:
img.copy().save(filepath, "PNG", pnginfo=metadata)
return True
except Exception:
logger.debug("Could not refresh placeholder timestamp for %s", filepath,
exc_info=True)
return False
def placeholder_age_seconds(filepath: Path) -> Optional[float]:
"""Seconds since a placeholder was written, or None if unknown."""
try:
with Image.open(filepath) as img:
stamped = img.info.get(PLACEHOLDER_MARKER)
if stamped and stamped != "1":
return max(0.0, time.time() - float(stamped))
except Exception:
pass
try:
return max(0.0, time.time() - filepath.stat().st_mtime)
except OSError:
return None
class LogoDownloader:
"""Centralized logo downloader for team logos from ESPN API."""
@@ -499,8 +590,10 @@ class LogoDownloader:
filename = f"{self.normalize_abbreviation(abbreviation)}.png"
filepath = Path(logo_dir) / filename
# Skip if already exists and not forcing download
if filepath.exists() and not force_download:
# A placeholder does not count as existing -- it is a previous
# failure, and a bulk pass is exactly where it should get another
# chance, subject to the same back-off as everywhere else.
if not should_attempt_download(filepath, force_download):
logger.debug(f"Skipping {display_name}: {filename} already exists")
continue
@@ -559,8 +652,9 @@ class LogoDownloader:
filename = f"{self.normalize_abbreviation(abbreviation)}.png"
filepath = Path(logo_dir) / filename
# Skip if already exists and not forcing download
if filepath.exists() and not force_download:
# Same eligibility rule as every other download site: a stale
# placeholder is a failed download, not a logo.
if not should_attempt_download(filepath, force_download):
logger.debug(f"Skipping {display_name} ({category}, {conference}): {filename} already exists")
continue
@@ -674,11 +768,16 @@ class LogoDownloader:
# Fallback without font
draw.text((16, 24), text, fill=(255, 255, 255, 255))
logo.save(filepath)
# Stamp it so a later run can tell this apart from a real logo and
# retry the download, instead of treating the file's existence as
# proof the logo was fetched.
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time()))
logo.save(filepath, "PNG", pnginfo=metadata)
# Set proper file permissions after saving
ensure_file_permissions(filepath, get_assets_file_mode())
logger.info(f"Created placeholder logo for {team_abbreviation} at {filepath}")
return True
@@ -770,9 +869,18 @@ def download_missing_logo(league: str, team_id: str, team_abbreviation: str, log
# Use the exact filepath that was passed in (respects config settings)
filepath = logo_path
if filepath.exists():
if filepath.exists() and not should_attempt_download(filepath):
# Either a real logo, or a placeholder too fresh to be worth retrying.
logger.debug(f"Logo already exists for {team_abbreviation} ({league})")
return True
if filepath.exists():
# A placeholder is a *failed* download wearing the real logo's
# filename. Treating it as "already exists" is what pinned a team to a
# grey box permanently after one transient failure.
logger.info(
"Logo for %s (%s) is a placeholder from a failed download; "
"retrying the real logo", team_abbreviation, league,
)
# Try to download the real logo first
logger.info(f"Attempting to download logo for {team_abbreviation} from {league}")