Files
LEDMatrix/src/logo_downloader.py
T
d9683e28be Codebase audit: fix shipping bugs, remove verified-dead code, repair doc drift, add regression guards (#438)
* fix(web): implement delete_cached so the font catalog cache actually invalidates

api_v3.py's font upload/delete handlers import delete_cached from
web_interface.cache, but the function was never defined. The surrounding
except ImportError silently swallowed the failure, so the fonts_catalog
cache entry survived uploads/deletes and newly uploaded fonts did not
appear until the TTL expired or the service restarted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(web): remove dead weather/stocks partial routes that returned 500

The partial dispatcher still routed 'weather' and 'stocks' to loaders
rendering v3/partials/weather.html and stocks.html — templates that no
longer exist since weather and stocks became store plugins. Requesting
either partial raised TemplateNotFound, which the catch-all turned into
a 500. No template or JS references these partials (the only 'weather'
hit in the front end is a plugin-store category filter option), so the
branches and both loader functions are removed; unknown partials now
fall through to the existing 404 handler.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(deps): align contradictory psutil/Flask-Limiter/freetype-py pins

requirements.txt's optional-install comment recommended psutil>=5.9,<6.0
while web_interface/requirements.txt hard-requires >=6.0,<7.0 — anyone
following the comment ends up with an unsatisfiable pair. The comment now
recommends the same range the web interface requires (all psutil APIs
used — Process, boot_time, cpu_percent, disk_usage, virtual_memory — are
stable in 6.x). Flask-Limiter gains the same <4.0 cap in both files and
freetype-py the same >=2.5.1 floor.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(config): add template keys the code already reads

display.hardware gains pixel_mapper_config, row_address_type,
multiplexing and panel_type (read at display_manager.py with these exact
fallbacks — users on non-standard panels previously had no way to
discover them from the template). vegas_scroll gains
frame_based_scrolling and scroll_delay, the only two of its 27 keys the
template omitted (read in src/vegas_mode/config.py). plugin_system gains
development_mode, which the web UI reads and writes but the template
never declared.

Every added value is byte-identical to the code-side .get() fallback, so
ConfigManager._migrate_config() merging these keys into existing user
configs cannot change behavior on any installed device.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(scripts): repair broken sys.path setup in utility scripts

clear_cache.py and download_nba_logos.py pointed sys.path at a 'src'
directory relative to the script's own folder (scripts/utils/src and
scripts/src — neither exists), so both crashed on import; they now insert
the project root and import via the src package like the other scripts.
debug_web_manual.py resolved 'project root' to scripts/debug/ instead of
two levels up. fix_nhl_cache.sh is removed: it used Python docstring
syntax in a bash script and invoked clear_nhl_cache.py, which does not
exist anywhere in the repo — it cannot ever have worked in its current
location.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: correct stale file:line references and the loader-fallback contradiction

CLAUDE.md and .cursorrules disagreed about plugin-directory fallback
behavior; the code (SchemaManager.get_schema_path) probes plugins/
BEFORE plugin-repos/, and the main discovery path has no fallback at
all — both files now describe the real behavior, preferring symbol names
over line numbers so the references rot slower. REST_API_REFERENCE.md
pointed at app.py:144/:607 for mounts that live at :199/:799 and counted
92 routes where there are 94. PLUGIN_ARCHITECTURE_SPEC.md's historical
banner gains a note that its example imports
(src/plugin_system/base_classes/*_plugin.py) never shipped — the real
base classes are src.base_classes.sports.SportsCore and
src.base_classes.hockey.Hockey.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: fix broken links, phantom script references, and stale CI description

Repairs every broken relative link in active docs (targets renamed or
archived long ago: PLUGIN_DEVELOPMENT.md -> PLUGIN_DEVELOPMENT_GUIDE.md,
API_REFERENCE.md -> REST_API_REFERENCE.md, PLUGIN_STORE_USER_GUIDE.md ->
PLUGIN_STORE_GUIDE.md, plugin_docs/ dir, TROUBLESHOOTING_QUICK_START.md,
and MIGRATION_GUIDE's README link that silently resolved to the docs
index instead of the project README). Replaces commands invoking scripts
that do not exist (scripts/update_stats.py, validate_registry.py,
check_updates.py, fix_permissions.sh) with the real tooling, and
rewrites HOW_TO_RUN_TESTS.md's CI section, which described a
security-audit workflow that was never committed and a pytest workflow
'queued to land' that landed long ago as test.yml.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: complete the docs index and refresh the web interface file tree

docs/README.md's own policy says every page must be linked from the
index, yet five weren't — including the entire skin system
(SKIN_SYSTEM.md, CREATING_SKINS.md), ADAPTIVE_LAYOUT.md,
plugin-safety-harness.md and SPORTS_UNIFICATION.md. Each is now listed
in the section it belongs to, and PLUGIN_ARCHITECTURE_SPEC.md is marked
historical in the index (the doc itself already carries the banner).
web_interface/README.md's static/v3 tree showed only app.css/app.js;
it now reflects the actual contents.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: remove dead modules confirmed unused in-repo and across all store plugins

- src/common/cli.py: imports a 'ledmatrix_common' package that exists
  nowhere (not in this repo, any requirements file, or the plugin
  monorepo), so it cannot ever have run; its README section claimed
  scripts/dev/* used it, which was also untrue.
- src/web_interface/logging_config.py: zero callers — the web app uses
  web_interface/logging_config.py (a different module), and nothing
  imports the src copy.
- handle_errors decorator in src/web_interface/error_handler.py: zero
  call sites (the module's response helpers stay — they are used).
- ConfigManager.get_clock_config(): reads a 'clock' config key that no
  longer exists anywhere; only caller was its own unit test.

Deliberately kept despite zero in-repo callers: DisplayError,
src/common/config_helper.py and display_helper.py — all documented as
plugin-facing API (docs/PLUGIN_ERROR_HANDLING.md, src/common/README.md),
and third-party plugins outside the official monorepo cannot be
enumerated. Verified against a fresh clone of ledmatrix-plugins (43
plugins): zero references to any removed symbol.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: remove manager-era NBA test files and one-off debug scripts

The four test_nba_*.py files imported nba_managers, leaderboard_manager
and odds_manager — top-level modules deleted when sports displays became
plugins — inside try/except blocks that swallowed the ImportError, so
they passed while exercising nothing. test_nba_data_structure.py and
debug_nba_api.py (a diagnostic script living in test/) made live ESPN
API calls rather than testing repo code. None were enrolled in CI.

scripts/debug/direct_fix_imports.py and check_imports.py were one-shot
artifacts that edited/inspected a hardcoded ~/LEDMatrix/web_interface/
app.py to fix an import problem solved long ago; nothing references
them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: remove generate_report.py, which aggregates artifacts of CI jobs that do not exist

The script's only function is to merge JSON artifacts
(bandit/semgrep/pip-audit/safety/gitleaks results) produced by a
security-audit workflow that was never committed —
.github/workflows/ has no such jobs, so there is nothing for it to
aggregate and no way to run it usefully. Its siblings stay:
prove_security.py and audit_plugins.py both run standalone (verified),
and .codacy.yml stays because the Codacy service (README badge) reads it
server-side without a workflow file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(web): load the three widget scripts store plugins already declare

time-picker.js, file-upload-single.js and plugin-file-manager.js
register widgets that installed store plugins reference in their config
schemas (countdown uses x-widget: time-picker and file-upload-single;
of-the-day uses plugin-file-manager), but base.html never included the
scripts. plugin_config.html renders such fields as an empty container
that polls LEDMatrixWidgets.get(...) on a 50ms loop forever, so those
plugin config fields appeared permanently blank. The audit initially
flagged these files as dead code; the monorepo cross-check proved the
opposite — they were unreachable, not unused.

example-color-picker.js (the documented custom-widget example) gains an
explicit warning that including it in base.html would shadow the
built-in color-picker widget.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: drop the legacy youtube block from the secrets template

No code reads a top-level youtube secrets key: the youtube-stats plugin
receives its API key namespaced under its own plugin id (declared via
x-secret in its config schema), like every other store plugin. The key
survives only in state_reconciliation.py's non-plugin-key exclusion set,
which stays — existing installs still carry the key in their generated
config_secrets.json, and the exclusion prevents it from being
misclassified as a plugin config. New installs simply stop being asked
for a YouTube API key they have nowhere to use.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore(deps): remove packages nothing imports, declare direct imports, move mypy to test deps

Removed from requirements.txt: python-socketio, python-engineio,
websockets, websocket-client — zero imports anywhere in this repo, and
the one store plugin that needs Socket.IO (ledmatrix-music) declares it
in its own requirements.txt, which the plugin store installs. Removed
the same quartet plus timezonefinder, geopy, google-auth-oauthlib,
google-auth-httplib2, google-api-python-client, unidecode, icalevents,
python-dateutil, flask-wtf and the werkzeug pin from
web_interface/requirements.txt — all leftovers from the deleted built-in
weather/calendar/music displays (flask-wtf was doubly dead: app.py
explicitly disables CSRF and sets csrf=None). scripts/
install_dependencies_apt.py, which mirrors these lists for the
first-time installer, drops the same packages.

Added: urllib3 (imported directly in four core modules), jinja2 and
markupsafe (imported directly in pages_v3.py) — previously reachable
only as transitives. mypy moves from runtime requirements to
requirements-test.txt.

Verified in a fresh venv: all four requirements files co-install, pip
check is clean, the full CI-enrolled suite (907 tests) and a Flask boot
smoke pass with the trimmed dependency set.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* refactor: single canonical DateTimeEncoder

src/cache_manager.py and src/cache/disk_cache.py each defined an
identical DateTimeEncoder (datetime -> ISO-8601). The disk_cache copy is
the only one actually used for serialization; cache_manager now
re-exports it instead of defining a twin, so the two can never silently
diverge. Import compatibility is preserved — from src.cache_manager
import DateTimeEncoder still works and is the same class object.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs(code): document deliberate duplicates instead of merging them

The audit surfaced several near-duplicate implementations that turned
out to be either deliberate forks or behaviorally different — merging
any of them would risk changing behavior on installed devices, so each
now carries an explicit comment stating the relationship:

- VisualDisplayManager: headless fork of DisplayManager; header now
  lists the ~15 mirrored methods and warns that DisplayManager changes
  must be mirrored.
- normalize_abbreviation: LogoDownloader's version (called directly by
  nine scoreboard plugins) replaces filesystem-unsafe characters;
  LogoHelper's strips spaces. Logo filenames on existing installs
  depend on both behaviors staying put.
- The two PluginTestBase classes: the shipped one is plugin-author
  API, the repo's own richer harness lives in test/plugins/ — now
  cross-referenced.

Also verified (no change needed): ConfigManager's backup/rollback
methods genuinely delegate to AtomicConfigManager, and SportsCore
already delegates _read_bdf_native_size to FontManager.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: add a unified configuration reference

There was no single place documenting what lives in config.json —
display.* keys were scattered across README sections, vegas_scroll lived
in ADVANCED_FEATURES.md, and dim_schedule, display.double_sided,
sync.follower_position, plugin_system.development_mode and the four
newly-templated hardware keys were documented nowhere. CONFIG_REFERENCE.md
now lists every template key plus the code-read-only keys, each with
type, default, and the code location that reads it, and explains the
secrets file's plugin-id namespacing. Linked from the docs index.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: bring the README's feature tour into the plugin era

The Core Features section still presented clock/weather/sports/stocks/
music displays as built into the project, when all of them are store
plugins installed from the ledmatrix-plugins monorepo — only
starlark-apps and web-ui-info ship in this repo. The intro now says so
(the showcase itself is unchanged; those are real displays available in
the store). The display_durations reference drops its built-in-calendar
example in favor of plugin-id keys, and the Configuration section links
the new CONFIG_REFERENCE.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: archive the custom-icons status report, cross-link config docs, document assets/

PLUGIN_CUSTOM_ICONS_FEATURE.md was a 'What Was Implemented' status
report duplicating the actual guide (PLUGIN_CUSTOM_ICONS.md) — moved to
docs/archive/ per the docs index's own policy. The overlapping
plugin-config docs keep their content but PLUGIN_CONFIG_ARCHITECTURE.md
now states up front which doc is canonical for which purpose.

assets/README.md is new and load-bearing: assets/stocks, weather,
news_logos and broadcast_logos have zero references in this repo's code,
which makes them look deletable — but store plugins (ledmatrix-stocks,
ledmatrix-weather, news, odds-ticker) resolve those exact paths at
runtime against the install directory. The README records that evidence
so a future cleanup doesn't break installed plugins.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* test: add regression guards for the bug classes fixed in this PR

Three lightweight static checks, all enrolled in CI's unit-test
allowlist along with the new web-cache test:

- test_template_targets.py: every literal render_template() target must
  exist (would have caught the weather/stocks partial 500s at commit
  time).
- test_widget_scripts.py: every widget JS file must be script-included
  in base.html or explicitly allowlisted with a reason (would have
  caught the unloaded time-picker/file-upload-single/plugin-file-manager
  widgets), and allowlisted files must NOT be included (prevents the
  example widget from shadowing the real color-picker).
- test_doc_links.py: relative markdown links in active docs must
  resolve (docs/archive/ exempt).

Each guard was verified to fail against the pre-PR tree and pass now.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(deps): restore the werkzeug version floor

Commit 1ec22db removed the werkzeug>=3.1.6,<4.0.0 pin along with the
genuinely-unused packages, but this one was a version floor on Flask's
transitive dependency, not a phantom: Flask 3.1.3 itself only requires
werkzeug>=3.1.0, so dropping the pin let fresh installs resolve
3.1.0-3.1.5. Restored with a comment explaining why it exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(web): route pixel_mapper_config into display.hardware; guard time-picker registration

pixel_mapper_config was the only display.hardware key absent from both
the display_fields detection allowlist and the hardware write loop in
the settings save path. No form posts it today, but if one ever did the
key would fall through to the generic handler and land at the TOP level
of config.json — where state_reconciliation would mistake it for a
missing plugin id and loop auto-repair attempts (the failure class the
'github'/'youtube' exclusion comment documents). It now round-trips
into display.hardware like its siblings.

time-picker.js gains the same LEDMatrixWidgets-undefined guard its two
sibling widgets already have; correct today only via defer ordering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: align installer leftovers with the dependency cleanup

first_time_install.sh's fallback secrets heredoc (used only when the
template is missing) still wrote the legacy youtube block — now matches
the template (github only). install_dependencies_apt.py drops the
IMPORT_NAME_MAP entries for packages no longer in its install lists and
a stale google-api reference in a docstring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix: address CodeRabbit review findings

Verified each finding against the code; fixes for the valid ones:

- install_dependencies_apt.py: the installer listed 'freetype', but the
  declared dependency is freetype-py — an apt miss would pip-install the
  wrong PyPI package. Now installs freetype-py with an import-name
  mapping (pre-existing bug, surfaced by the review).
- api_v3.py: pixel_mapper_config is validated as a string before being
  saved to display.hardware (JSON callers could previously store an
  object/list the matrix library can't use).
- .cursorrules: the Plugin Loading Process and File Organization
  sections still said discovery scans plugins/ — now consistent with the
  corrected overview (configured directory, default plugin-repos/).
- README.md: removed the stale '(except the core calendar)' claim — no
  core calendar exists in src/ — and qualified the plugin inventory
  (official plugins in the monorepo; third-party from their own repos).
- CONFIG_REFERENCE.md: hardware_mapping now shows the code fallback
  (adafruit-hat-pwm) alongside the template value.
- PLUGIN_REGISTRY_SETUP_GUIDE.md: check_plugin.py takes --plugin, not a
  positional id.
- scripts/fix_perms/fix_*.sh: exec bits set so the documented
  'sudo ./...' invocations work.
- Guard tests hardened: template guard now catches multi-line
  render_template() calls; widget guard parses actual <script> src
  values and fails if the widgets dir goes missing; type hints and
  docstrings added per repo coding guidelines.

Skipped with reasons (noted on the PR): limit_refresh_rate_hz 100-vs-90
is documented as intentional in CONFIG_REFERENCE.md; the psutil comment
already names the enforcing manifest; docs/archive/ findings are out of
scope per the docs policy (archive may rot).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: remove Cursor IDE tooling, consolidate its guidance into CLAUDE.md

The maintainer no longer uses Cursor. .cursorrules, .cursorignore and the
.cursor/ tree (rules, plugin templates, a parallel 751-line plugins
guide) are removed; measurement showed near-zero literal overlap risk —
the canonical content already lives in docs/. Unique guidance worth
keeping moved before deletion:

- CLAUDE.md gains the dev workflow (dev_plugin_setup.sh, dev_server.py,
  run.py -e, check_plugin.py), the plugin-secrets namespacing contract,
  and the no-draw_image()/paste-onto-PIL pitfall.
- PLUGIN_DEVELOPMENT_GUIDE.md absorbs the plugin version-management
  rules (pre-push hook install, SKIP_TAG, version resolution order) that
  its own text previously linked out to .cursorrules for.
- The one completed plan doc (.cursor/plans/) is archived to
  docs/archive/ per the docs policy rather than deleted.

One of the deleted rule files (sports-managers.mdc) targeted
src/*_managers.py globs that have matched nothing since the plugin
migration.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: second-pass cleanup — dead installer branch, broken script, orphaned JS, misfiled test deps

- first_time_install.sh: removed the pip fallback branch that installed
  from requirements_web_v2.txt — a file that has not existed since the
  v2 web interface was removed (the branch always printed its own
  'not found; skipping' warning).
- scripts/remove_plugin_backups.sh deleted: its PROJECT_ROOT resolved to
  the repo's PARENT directory, and its verify_submodules() checks for
  plugin submodules from an era before plugins moved to the store — it
  could never have worked from its current location.
- plugins_manager.js: removed three functions with zero call sites
  anywhere (addKeyValuePair, formatCommit, togglePasswordVisibility) —
  verified against all templates, all JS, and the dynamic window[name]
  dispatch sites, which resolve widget-registry keys only. Also replaced
  base.html's misleading 'Legacy ... during migration' label: the file
  is deliberately loaded last and provides the LIVE implementations of
  seven window.* plugin actions that shadow same-named definitions in
  app.js/app-shell.js.
- pytest/pytest-cov/pytest-mock moved from runtime requirements.txt to
  requirements-test.txt (CI already installs both files; the installer's
  line-by-line loop simply installs three fewer packages on devices; no
  store plugin declares pytest). HOW_TO_RUN_TESTS.md updated.
- scripts/add_defaults_to_schemas.py and analyze_plugin_schemas.py
  scanned the empty legacy plugins/ dir — now scan plugin-repos/.

Verified: fresh venv installs all four requirements files with pip check
clean and pytest available; bash -n on the installer; node --check on
the JS; widget/cache guard tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: correct semantically stale content across the user and developer guides

A second-pass content audit checked the guides' substantive claims
against the code (the first pass only fixed mechanical drift). Fixes:

- GETTING_STARTED: described booting a prebuilt SD image and seeing
  default clock/weather plugins — neither exists. Now documents the real
  install (Pi OS Lite + one-shot installer / first_time_install.sh) and
  that displays come from the Plugin Store. Duration and ordering
  instructions moved to the Rotation tab where the controls actually
  live.
- WEB_INTERFACE_GUIDE: three whole tabs were undocumented (Rotation,
  Backup & Restore, Tools) and the Display tab's Vegas Scroll section
  was unmentioned. Fonts overrides are per display element (not per
  plugin); Logs has an Auto-scroll checkbox (not a Pause button); the
  aspirational keyboard-shortcut list and no-JS claim removed.
- TROUBLESHOOTING: the hand-written service-file template (wrong user,
  wrong ExecStart, dropped the autostart gate) replaced with the real
  systemd/ units + install scripts; recovery steps no longer copy
  placeholder units verbatim; WiFi curl endpoint corrected to /api/v3/;
  cache-clearing advice now targets the real cache locations.
- ADVANCED_FEATURES: removed a false claim that CacheManager has no
  delete(); fixed two example snippets that raise TypeError
  (BackgroundDataService and get_config_file_mode signatures); fixed
  cache paths, a 5-minute TTL that is actually 1 hour, and the vegas
  table now links the complete 26-key reference.
- EMULATOR_SETUP_GUIDE: documented run.py flags that don't exist
  (--plugin/--test-plugins) removed in favor of dev_server.py and
  check_plugin.py; shipped emulator config values corrected (browser
  adapter default on :8888, not pygame).
- PLUGIN_QUICK_REFERENCE: drag-and-drop reordering is shipped, not
  'not yet supported'; discovery-fallback and registry-repo claims
  corrected. PLUGIN_API_REFERENCE: get_vegas_segment_width returns
  panels, not pixels. CONTRIBUTING: the repo uses flake8/mypy/bandit
  pre-commit hooks, not black/ruff, and tests need requirements-test.txt.
- SKIN_SYSTEM/DEVELOPER_QUICK_REFERENCE: stale module paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix: raise the two new dependency floors past their CVEs, silence a deliberate re-export

All three of these were introduced by this PR, which is what makes them
worth fixing here rather than deferring.

`urllib3` and `jinja2` were added to the requirements so that direct
imports stop relying on transitives — right call, but both floors were
set to the version that introduced the API rather than a version that is
safe to install. `urllib3>=1.26.0` sits below roughly ten CVEs including
a decompression-bomb safeguard bypass, and `jinja2>=3.1.0` below five
including two sandbox breakouts. Raised to 2.7.0 and 3.1.6, which is what
a working device already runs, so no install is disturbed. The comments
now say the floor is a security floor, since the next person to read
"imported directly" would otherwise reasonably lower it again.

This is the same reasoning the PR already applied to werkzeug; these two
just missed it.

The `DateTimeEncoder` import in cache_manager is unused on purpose — the
canonical class moved to src.cache.disk_cache and this re-export keeps
the documented import path working. flake8 cannot see intent, so it gets
an explicit `# noqa: F401` rather than being removed and quietly breaking
anything importing it from here. Verified the re-export still resolves to
the same object and still serialises datetimes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix: address CodeRabbit re-review — installer robustness and doc lint

- install_dependencies_apt.py: an import-only check let Debian
  Bookworm's python3-freetype 2.3.0 satisfy the freetype-py>=2.5.1 pin.
  check_package_installed() now verifies the installed freetype-py
  version, and an apt install that lands below the minimum falls through
  to pip instead of counting as success.
- first_time_install.sh: the .web_deps_installed marker was created even
  when the smart installer failed, so re-runs skipped installation with
  dependencies missing. The marker is now created only on success.
- CONTRIBUTING.md: document installing the pre-commit CLI before
  'pre-commit install' (the requirements files don't provide it).
- Doc lint: fence language on the on-demand cache example (MD040),
  blockquote continuation in GETTING_STARTED (MD028), and the
  suppress_adapter_load_errors key removed from the emulator debug
  example to match the options table.

Skipped one finding with reason (noted on the PR): the per-plugin
display_duration field in PLUGIN_QUICK_REFERENCE's example is not
obsolete — BasePlugin.get_display_duration() reads it and
PLUGIN_CONFIG_CORE_PROPERTIES.md documents it as a core property;
display.display_durations is a per-mode override, not a replacement.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-06 14:04:23 -04:00

816 lines
36 KiB
Python

#!/usr/bin/env python3
"""
Centralized logo downloader utility for automatically fetching team logos from ESPN API.
This module provides functionality to download missing team logos for various sports leagues,
with special support for FCS teams and other NCAA divisions.
"""
import os
import re
import time
import logging
import requests
import json
from typing import Dict, List, Optional, Tuple
from pathlib import Path
from PIL import Image, ImageDraw, ImageFont
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from src.common.permission_utils import (
ensure_directory_permissions,
ensure_file_permissions,
get_assets_dir_mode,
get_assets_file_mode
)
logger = logging.getLogger(__name__)
class LogoDownloader:
"""Centralized logo downloader for team logos from ESPN API."""
# ESPN API endpoints for different sports/leagues
API_ENDPOINTS = {
'nfl': 'https://site.api.espn.com/apis/site/v2/sports/football/nfl/teams',
'nba': 'https://site.api.espn.com/apis/site/v2/sports/basketball/nba/teams',
'mlb': 'https://site.api.espn.com/apis/site/v2/sports/baseball/mlb/teams',
'nhl': 'https://site.api.espn.com/apis/site/v2/sports/hockey/nhl/teams',
'ncaa_fb': 'https://site.api.espn.com/apis/site/v2/sports/football/college-football/teams',
'ncaa_fb_all': 'https://site.api.espn.com/apis/site/v2/sports/football/college-football/teams', # Includes FCS
'fcs': 'https://site.api.espn.com/apis/site/v2/sports/football/college-football/teams', # FCS teams from same endpoint
'ncaam_basketball': 'https://site.api.espn.com/apis/site/v2/sports/basketball/mens-college-basketball/teams',
'ncaam': 'https://site.api.espn.com/apis/site/v2/sports/basketball/mens-college-basketball/teams', # Alias for basketball plugin
'ncaaw_basketball': 'https://site.api.espn.com/apis/site/v2/sports/basketball/womens-college-basketball/teams',
'ncaaw': 'https://site.api.espn.com/apis/site/v2/sports/basketball/womens-college-basketball/teams', # Alias for basketball plugin
'ncaa_baseball': 'https://site.api.espn.com/apis/site/v2/sports/baseball/college-baseball/teams',
'ncaam_hockey': 'https://site.api.espn.com/apis/site/v2/sports/hockey/mens-college-hockey/teams',
'ncaaw_hockey': 'https://site.api.espn.com/apis/site/v2/sports/hockey/womens-college-hockey/teams',
'ncaam_lacrosse': 'https://site.api.espn.com/apis/site/v2/sports/lacrosse/mens-college-lacrosse/teams',
'ncaaw_lacrosse': 'https://site.api.espn.com/apis/site/v2/sports/lacrosse/womens-college-lacrosse/teams',
# Soccer leagues
'soccer_eng.1': 'https://site.api.espn.com/apis/site/v2/sports/soccer/eng.1/teams',
'soccer_esp.1': 'https://site.api.espn.com/apis/site/v2/sports/soccer/esp.1/teams',
'soccer_ger.1': 'https://site.api.espn.com/apis/site/v2/sports/soccer/ger.1/teams',
'soccer_ita.1': 'https://site.api.espn.com/apis/site/v2/sports/soccer/ita.1/teams',
'soccer_fra.1': 'https://site.api.espn.com/apis/site/v2/sports/soccer/fra.1/teams',
'soccer_por.1': 'https://site.api.espn.com/apis/site/v2/sports/soccer/por.1/teams',
'soccer_uefa.champions': 'https://site.api.espn.com/apis/site/v2/sports/soccer/uefa.champions/teams',
'soccer_uefa.europa': 'https://site.api.espn.com/apis/site/v2/sports/soccer/uefa.europa/teams',
'soccer_usa.1': 'https://site.api.espn.com/apis/site/v2/sports/soccer/usa.1/teams'
}
# Directory mappings for different leagues
LOGO_DIRECTORIES = {
'nfl': 'assets/sports/nfl_logos',
'nba': 'assets/sports/nba_logos',
'wnba': 'assets/sports/wnba_logos',
'mlb': 'assets/sports/mlb_logos',
'nhl': 'assets/sports/nhl_logos',
# NCAA sports use same directory
'ncaa_fb': 'assets/sports/ncaa_logos',
'ncaa_fb_all': 'assets/sports/ncaa_logos',
'fcs': 'assets/sports/ncaa_logos',
'ncaam_basketball': 'assets/sports/ncaa_logos',
'ncaam': 'assets/sports/ncaa_logos', # Alias for basketball plugin
'ncaaw_basketball': 'assets/sports/ncaa_logos',
'ncaaw': 'assets/sports/ncaa_logos', # Alias for basketball plugin
'ncaa_baseball': 'assets/sports/ncaa_logos',
'ncaam_hockey': 'assets/sports/ncaa_logos',
'ncaaw_hockey': 'assets/sports/ncaa_logos',
'ncaam_lacrosse': 'assets/sports/ncaa_logos',
'ncaaw_lacrosse': 'assets/sports/ncaa_logos',
# Soccer leagues - all use the same soccer_logos directory
'soccer_eng.1': 'assets/sports/soccer_logos',
'soccer_esp.1': 'assets/sports/soccer_logos',
'soccer_ger.1': 'assets/sports/soccer_logos',
'soccer_ita.1': 'assets/sports/soccer_logos',
'soccer_fra.1': 'assets/sports/soccer_logos',
'soccer_por.1': 'assets/sports/soccer_logos',
'soccer_uefa.champions': 'assets/sports/soccer_logos',
'soccer_uefa.europa': 'assets/sports/soccer_logos',
'soccer_usa.1': 'assets/sports/soccer_logos'
}
def __init__(self, request_timeout: int = 30, retry_attempts: int = 3):
"""Initialize the logo downloader with HTTP session and retry logic."""
self.request_timeout = request_timeout
self.retry_attempts = retry_attempts
# Set up session with retry logic
self.session = requests.Session()
retry_strategy = Retry(
total=retry_attempts,
backoff_factor=1,
status_forcelist=[429, 500, 502, 503, 504],
allowed_methods=["GET", "HEAD", "OPTIONS"]
)
adapter = HTTPAdapter(max_retries=retry_strategy)
self.session.mount("https://", adapter)
self.session.mount("http://", adapter)
# Set up headers
self.headers = {
'User-Agent': 'LEDMatrix/1.0 (https://github.com/yourusername/LEDMatrix; contact@example.com)',
'Accept': 'application/json',
'Accept-Language': 'en-US,en;q=0.9',
'Accept-Encoding': 'gzip, deflate, br',
'Connection': 'keep-alive'
}
@staticmethod
def normalize_abbreviation(abbreviation: str) -> str:
"""Normalize team abbreviation for consistent filename usage.
Public API: sports scoreboard plugins call this directly.
NOTE: LogoHelper.normalize_abbreviation (src/common/logo_helper.py)
is a deliberately different variant (strips spaces, fewer character
replacements) — keep both behaviors stable; logo filenames on
existing installs depend on them.
"""
# Handle special characters that can cause filesystem issues
normalized = abbreviation.upper()
# Replace problematic characters with safe alternatives
normalized = normalized.replace('&', 'AND')
normalized = normalized.replace('/', '_')
normalized = normalized.replace('\\', '_')
normalized = normalized.replace(':', '_')
normalized = normalized.replace('*', '_')
normalized = normalized.replace('?', '_')
normalized = normalized.replace('"', '_')
normalized = normalized.replace('<', '_')
normalized = normalized.replace('>', '_')
normalized = normalized.replace('|', '_')
return normalized
@staticmethod
def get_logo_filename_variations(abbreviation: str) -> list:
"""Get possible filename variations for a team abbreviation."""
variations = []
original = abbreviation.upper()
normalized = LogoDownloader.normalize_abbreviation(abbreviation)
# Add original and normalized versions
variations.extend([f"{original}.png", f"{normalized}.png"])
# Special handling for known cases
if original == 'TA&M':
# TA&M has a file named TA&M.png, but normalize creates TAANDM.png
variations = [f"{original}.png", f"{normalized}.png"]
return variations
# Allowlist for league names used in filesystem paths: alphanumerics, underscores, dashes only
_SAFE_LEAGUE_RE = re.compile(r'^[a-z0-9_-]+$')
def get_logo_directory(self, league: str) -> str:
"""Get the logo directory for a given league."""
directory = LogoDownloader.LOGO_DIRECTORIES.get(league)
if not directory:
# Custom soccer leagues share the same logo directory as predefined ones
if league.startswith('soccer_'):
directory = 'assets/sports/soccer_logos'
else:
# Validate league before using it in a filesystem path
if not self._SAFE_LEAGUE_RE.match(league):
logger.warning(f"Rejecting unsafe league name for directory construction: {league!r}")
raise ValueError(f"Unsafe league name: {league!r}")
directory = f'assets/sports/{league}_logos'
path = Path(directory)
if not path.is_absolute():
project_root = Path(__file__).resolve().parents[1]
path = (project_root / path).resolve()
return str(path)
def ensure_logo_directory(self, logo_dir: str | Path) -> bool:
"""Ensure the logo directory exists, create if necessary."""
path = Path(logo_dir)
try:
# Create directory with proper permissions
ensure_directory_permissions(path, get_assets_dir_mode())
# Check if we can actually write to the directory
test_file = path / '.write_test'
try:
with open(test_file, 'w') as f:
f.write('test')
test_file.unlink(missing_ok=True)
logger.debug(f"Directory {path} is writable")
return True
except PermissionError:
logger.error(f"Permission denied: Cannot write to directory {path}")
logger.error("Please run: sudo ./scripts/fix_perms/fix_assets_permissions.sh")
return False
except Exception as e:
logger.error(f"Failed to test write access to directory {path}: {e}")
return False
except Exception as e:
logger.error(f"Failed to create logo directory {path}: {e}")
return False
def download_logo(self, logo_url: str, filepath: Path, team_abbreviation: str) -> bool:
"""Download a single logo from URL and save to filepath."""
try:
response = self.session.get(logo_url, headers=self.headers, timeout=self.request_timeout)
response.raise_for_status()
# Verify it's actually an image
content_type = response.headers.get('content-type', '').lower()
if not any(img_type in content_type for img_type in ['image/png', 'image/jpeg', 'image/jpg', 'image/gif']):
logger.warning(f"Downloaded content for {team_abbreviation} is not an image: {content_type}")
return False
with open(filepath, 'wb') as f:
f.write(response.content)
# Verify and convert the downloaded image to RGBA format
try:
with Image.open(filepath) as img:
# Convert to RGBA to avoid PIL warnings about palette images with transparency
if img.mode in ('P', 'LA', 'L'):
# Convert palette or grayscale images to RGBA
img = img.convert('RGBA')
elif img.mode == 'RGB':
# Convert RGB to RGBA (add alpha channel)
img = img.convert('RGBA')
elif img.mode != 'RGBA':
# For any other mode, convert to RGBA
img = img.convert('RGBA')
# Save the converted image
img.save(filepath, 'PNG')
# Set proper file permissions after saving
ensure_file_permissions(filepath, get_assets_file_mode())
logger.info(f"Successfully downloaded and converted logo for {team_abbreviation} -> {filepath.name}")
return True
except Exception as e:
logger.error(f"Downloaded file for {team_abbreviation} is not a valid image or conversion failed: {e}")
try:
os.remove(filepath) # Remove invalid file
except OSError:
pass
return False
except PermissionError as e:
logger.error(f"Permission denied downloading logo for {team_abbreviation}: {e}")
logger.error("Please run: sudo ./scripts/fix_perms/fix_assets_permissions.sh")
return False
except requests.exceptions.RequestException as e:
logger.error(f"Failed to download logo for {team_abbreviation}: {e}")
return False
except Exception as e:
logger.error(f"Unexpected error downloading logo for {team_abbreviation}: {e}")
return False
# Allowlist for the league_code segment interpolated into ESPN API URLs
_SAFE_LEAGUE_CODE_RE = re.compile(r'^[a-z0-9_-]+$')
def _resolve_api_url(self, league: str) -> Optional[str]:
"""Resolve the ESPN API teams URL for a league, with dynamic fallback for custom soccer leagues."""
api_url = self.API_ENDPOINTS.get(league)
if not api_url and league.startswith('soccer_'):
league_code = league[len('soccer_'):]
if not self._SAFE_LEAGUE_CODE_RE.match(league_code):
logger.warning(f"Rejecting unsafe league_code for ESPN URL construction: {league_code!r}")
return None
api_url = f'https://site.api.espn.com/apis/site/v2/sports/soccer/{league_code}/teams'
logger.info(f"Using dynamic ESPN endpoint for custom soccer league: {league}")
return api_url
def fetch_teams_data(self, league: str) -> Optional[Dict]:
"""Fetch team data from ESPN API for a specific league."""
api_url = self._resolve_api_url(league)
if not api_url:
logger.error(f"No API endpoint configured for league: {league}")
return None
try:
logger.info(f"Fetching team data for {league} from ESPN API...")
response = self.session.get(api_url, params={'limit':1000},headers=self.headers, timeout=self.request_timeout)
response.raise_for_status()
data = response.json()
logger.info(f"Successfully fetched team data for {league}")
return data
except requests.exceptions.RequestException as e:
logger.error(f"Error fetching team data for {league}: {e}")
return None
except json.JSONDecodeError as e:
logger.error(f"Error parsing JSON response for {league}: {e}")
return None
def fetch_single_team(self, league: str, team_id: str) -> Optional[Dict]:
"""Fetch team data from ESPN API for a specific league."""
api_url = self._resolve_api_url(league)
if not api_url:
logger.error(f"No API endpoint configured for league: {league}")
return None
try:
logger.info(f"Fetching team data for team {team_id} in {league} from ESPN API...")
response = self.session.get(f"{api_url}/{team_id}", headers=self.headers, timeout=self.request_timeout)
response.raise_for_status()
data = response.json()
logger.info(f"Successfully fetched team data for {team_id} in {league}")
return data
except requests.exceptions.RequestException as e:
logger.error(f"Error fetching team data for {team_id} in {league}: {e}")
return None
except json.JSONDecodeError as e:
logger.error(f"Error parsing JSON response for{team_id} in {league}: {e}")
return None
def extract_teams_from_data(self, data: Dict, league: str) -> List[Dict[str, str]]:
"""Extract team information from ESPN API response."""
teams = []
try:
sports = data.get('sports', [])
for sport in sports:
leagues_data = sport.get('leagues', [])
for league_data in leagues_data:
teams_data = league_data.get('teams', [])
for team_data in teams_data:
team_info = team_data.get('team', {})
abbreviation = team_info.get('abbreviation', '')
display_name = team_info.get('displayName', 'Unknown')
logos = team_info.get('logos', [])
if not abbreviation or not logos:
continue
# Get the default logo (first one is usually default)
logo_url = logos[0].get('href', '')
if not logo_url:
continue
# For NCAA football, try to determine if it's FCS or FBS
team_category = 'FBS' # Default
if league in ['ncaa_fb', 'ncaa_fb_all', 'fcs']:
# Check if this is an FCS team by looking at conference or other indicators
# ESPN API includes both FBS and FCS teams in the same endpoint
# We'll include all teams and let the user decide which ones to use
team_category = self._determine_ncaa_football_division(team_info, league_data)
teams.append({
'abbreviation': abbreviation,
'display_name': display_name,
'logo_url': logo_url,
'league': league,
'category': team_category,
'conference': league_data.get('name', 'Unknown')
})
logger.info(f"Extracted {len(teams)} teams for {league}")
return teams
except Exception as e:
logger.error(f"Error extracting teams for {league}: {e}")
return []
def _determine_ncaa_football_division(self, team_info: Dict, league_data: Dict) -> str:
"""Determine if an NCAA football team is FBS or FCS based on conference and other indicators."""
conference_name = league_data.get('name', '').lower()
# FBS Conferences (more comprehensive list)
fbs_conferences = {
'acc', 'american athletic', 'big 12', 'big ten', 'conference usa', 'c-usa',
'mid-american', 'mac', 'mountain west', 'pac-12', 'pac-10', 'sec',
'sun belt', 'independents', 'big east'
}
# FCS Conferences (more comprehensive list)
fcs_conferences = {
'big sky', 'big south', 'colonial athletic', 'caa', 'ivy league',
'meac', 'missouri valley', 'mvfc', 'northeast', 'nec',
'ohio valley', 'ovc', 'patriot league', 'pioneer football',
'southland', 'southern', 'southwestern athletic', 'swac',
'western athletic', 'wac', 'ncaa division i-aa'
}
# Also check for specific team indicators
team_abbreviation = team_info.get('abbreviation', '').upper()
# Known FBS teams that might be misclassified
known_fbs_teams = {
'ASU', 'ARIZ', 'ARK', 'AUB', 'BOIS', 'CSU', 'FLA', 'HAW', 'IDHO', 'USA'
}
# Check if it's a known FBS team first
if team_abbreviation in known_fbs_teams:
return 'FBS'
# Check conference names
if any(fbs_conf in conference_name for fbs_conf in fbs_conferences):
return 'FBS'
elif any(fcs_conf in conference_name for fcs_conf in fcs_conferences):
return 'FCS'
# If conference is just "NCAA - Football", we need to use other indicators
if conference_name == 'ncaa - football':
# Check team name for indicators of FCS (smaller schools, Division II/III)
team_name = team_info.get('displayName', '').lower()
fcs_indicators = ['college', 'university', 'state', 'tech', 'community']
# If it has typical FCS naming patterns and isn't a known FBS team
if any(indicator in team_name for indicator in fcs_indicators):
return 'FCS'
else:
return 'FBS'
# Default to FBS for unknown conferences
return 'FBS'
def _get_team_name_variations(self, abbreviation: str) -> List[str]:
"""Generate common variations of a team abbreviation for matching."""
variations = set()
abbr = abbreviation.upper()
variations.add(abbr)
# Add normalized version
variations.add(self.normalize_abbreviation(abbr))
# Common substitutions
substitutions = {
'&': ['AND', 'A'],
'A&M': ['TAMU', 'TA&M', 'TEXASAM'],
'STATE': ['ST', 'ST.'],
'UNIVERSITY': ['U', 'UNIV'],
'COLLEGE': ['C', 'COL'],
'TECHNICAL': ['TECH', 'T'],
'NORTHERN': ['NORTH', 'N'],
'SOUTHERN': ['SOUTH', 'S'],
'EASTERN': ['EAST', 'E'],
'WESTERN': ['WEST', 'W']
}
# Apply substitutions
for original, replacements in substitutions.items():
if original in abbr:
for replacement in replacements:
variations.add(abbr.replace(original, replacement))
variations.add(abbr.replace(original, '')) # Remove the word entirely
# Add common abbreviations for Texas A&M
if 'A&M' in abbr or 'TAMU' in abbr:
variations.update(['TAMU', 'TA&M', 'TEXASAM', 'TEXAS_A&M', 'TEXAS_AM'])
return list(variations)
def download_missing_logos_for_league(self, league: str, force_download: bool = False) -> Tuple[int, int]:
"""Download missing logos for a specific league."""
logger.info(f"Starting logo download for league: {league}")
# Get logo directory
logo_dir = self.get_logo_directory(league)
if not self.ensure_logo_directory(logo_dir):
logger.error(f"Failed to create logo directory for {league}")
return 0, 0
# Fetch team data
data = self.fetch_teams_data(league)
if not data:
logger.error(f"Failed to fetch team data for {league}")
return 0, 0
# Extract teams
teams = self.extract_teams_from_data(data, league)
if not teams:
logger.warning(f"No teams found for {league}")
return 0, 0
# Download missing logos
downloaded_count = 0
failed_count = 0
for team in teams:
abbreviation = team['abbreviation']
display_name = team['display_name']
logo_url = team['logo_url']
# Create filename
filename = f"{self.normalize_abbreviation(abbreviation)}.png"
filepath = Path(logo_dir) / filename
# Skip if already exists and not forcing download
if filepath.exists() and not force_download:
logger.debug(f"Skipping {display_name}: {filename} already exists")
continue
# Download logo
if self.download_logo(logo_url, filepath, display_name):
downloaded_count += 1
else:
failed_count += 1
# Small delay to be respectful to the API
time.sleep(0.1)
logger.info(f"Logo download complete for {league}: {downloaded_count} downloaded, {failed_count} failed")
return downloaded_count, failed_count
def download_all_ncaa_football_logos(self, include_fcs: bool = True, force_download: bool = False) -> Tuple[int, int]:
"""Download all NCAA football team logos including FCS teams."""
logger.info(f"Starting comprehensive NCAA football logo download (FCS: {include_fcs})")
# Use the comprehensive NCAA football endpoint
league = 'ncaa_fb_all'
logo_dir = self.get_logo_directory(league)
if not self.ensure_logo_directory(logo_dir):
logger.error(f"Failed to create logo directory for {league}")
return 0, 0
# Fetch team data
data = self.fetch_teams_data(league)
if not data:
logger.error(f"Failed to fetch team data for {league}")
return 0, 0
# Extract teams
teams = self.extract_teams_from_data(data, league)
if not teams:
logger.warning(f"No teams found for {league}")
return 0, 0
# Filter teams based on FCS inclusion
if not include_fcs:
teams = [team for team in teams if team.get('category') == 'FBS']
logger.info(f"Filtered to FBS teams only: {len(teams)} teams")
# Download missing logos
downloaded_count = 0
failed_count = 0
for team in teams:
abbreviation = team['abbreviation']
display_name = team['display_name']
logo_url = team['logo_url']
category = team.get('category', 'Unknown')
conference = team.get('conference', 'Unknown')
# Create filename
filename = f"{self.normalize_abbreviation(abbreviation)}.png"
filepath = Path(logo_dir) / filename
# Skip if already exists and not forcing download
if filepath.exists() and not force_download:
logger.debug(f"Skipping {display_name} ({category}, {conference}): {filename} already exists")
continue
# Download logo
if self.download_logo(logo_url, filepath, display_name):
downloaded_count += 1
logger.info(f"Downloaded {display_name} ({category}, {conference}) -> {filename}")
else:
failed_count += 1
logger.warning(f"Failed to download {display_name} ({category}, {conference})")
# Small delay to be respectful to the API
time.sleep(0.1)
logger.info(f"Comprehensive NCAA football logo download complete: {downloaded_count} downloaded, {failed_count} failed")
return downloaded_count, failed_count
def download_missing_logo_for_team(self, league: str, team_id: str, team_abbreviation: str, logo_path: Path) -> bool:
"""Download a specific team's logo if it's missing."""
# Ensure the logo directory exists and is writable
logo_dir = str(logo_path.parent)
if not self.ensure_logo_directory(logo_dir):
logger.error(f"Cannot download logo for {team_abbreviation}: directory {logo_dir} is not writable")
return False
# Fetch team data to find the logo URL
data = self.fetch_single_team(league, team_id)
if not data:
return False
try:
logo_url = data["team"]["logos"][0]["href"]
except KeyError:
return False
# Download the logo
success = self.download_logo(logo_url, logo_path, team_abbreviation)
if success:
time.sleep(0.1) # Small delay
return success
def download_all_missing_logos(self, leagues: List[str] | None = None, force_download: bool = False) -> Dict[str, Tuple[int, int]]:
"""Download missing logos for all specified leagues."""
if leagues is None:
leagues = list(self.API_ENDPOINTS.keys())
results = {}
total_downloaded = 0
total_failed = 0
for league in leagues:
if not self._resolve_api_url(league):
logger.warning(f"Skipping unknown league: {league}")
continue
downloaded, failed = self.download_missing_logos_for_league(league, force_download)
results[league] = (downloaded, failed)
total_downloaded += downloaded
total_failed += failed
logger.info(f"Overall logo download results: {total_downloaded} downloaded, {total_failed} failed")
return results
def create_placeholder_logo(self, team_abbreviation: str, logo_dir: str) -> bool:
"""Create a placeholder logo when real logo cannot be downloaded."""
try:
# Ensure the logo directory exists
if not self.ensure_logo_directory(logo_dir):
logger.error(f"Failed to create logo directory: {logo_dir}")
return False
filename = f"{self.normalize_abbreviation(team_abbreviation)}.png"
filepath = Path(logo_dir) / filename
# Check if we can write to the directory
try:
# Test write permissions by creating a temporary file
test_file = filepath.parent / "test_write.tmp"
test_file.touch()
test_file.unlink() # Remove the test file
except PermissionError:
logger.error(f"Permission denied: Cannot write to directory {logo_dir}")
return False
except Exception as e:
logger.error(f"Directory access error for {logo_dir}: {e}")
return False
# Create a simple placeholder logo
logo = Image.new('RGBA', (64, 64), (100, 100, 100, 255)) # Gray background
draw = ImageDraw.Draw(logo)
# Try to load a font, fallback to default
try:
font = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 12)
except (OSError, IOError):
try:
font = ImageFont.load_default()
except (OSError, IOError):
font = None
# Draw team abbreviation
text = team_abbreviation
if font:
# Center the text
bbox = draw.textbbox((0, 0), text, font=font)
text_width = bbox[2] - bbox[0]
text_height = bbox[3] - bbox[1]
x = (64 - text_width) // 2
y = (64 - text_height) // 2
draw.text((x, y), text, font=font, fill=(255, 255, 255, 255))
else:
# Fallback without font
draw.text((16, 24), text, fill=(255, 255, 255, 255))
logo.save(filepath)
# Set proper file permissions after saving
ensure_file_permissions(filepath, get_assets_file_mode())
logger.info(f"Created placeholder logo for {team_abbreviation} at {filepath}")
return True
except Exception as e:
logger.error(f"Failed to create placeholder logo for {team_abbreviation}: {e}")
return False
def convert_image_to_rgba(self, filepath: Path) -> bool:
"""Convert an image file to RGBA format to avoid PIL warnings."""
try:
with Image.open(filepath) as img:
if img.mode != 'RGBA':
# Convert to RGBA
converted_img = img.convert('RGBA')
converted_img.save(filepath, 'PNG')
logger.debug(f"Converted {filepath.name} from {img.mode} to RGBA")
return True
else:
logger.debug(f"{filepath.name} is already in RGBA format")
return True
except Exception as e:
logger.error(f"Failed to convert {filepath.name} to RGBA: {e}")
return False
def convert_all_logos_to_rgba(self, league: str) -> Tuple[int, int]:
"""Convert all logos in a league directory to RGBA format."""
logo_dir = Path(self.get_logo_directory(league))
if not logo_dir.exists():
logger.warning(f"Logo directory does not exist: {logo_dir}")
return 0, 0
converted_count = 0
failed_count = 0
for logo_file in logo_dir.glob("*.png"):
if self.convert_image_to_rgba(logo_file):
converted_count += 1
else:
failed_count += 1
logger.info(f"Converted {converted_count} logos to RGBA format for {league}, {failed_count} failed")
return converted_count, failed_count
# Helper function to map soccer league codes to logo downloader format
def get_soccer_league_key(league_code: str) -> str:
"""
Map soccer league codes to logo downloader format.
Args:
league_code: Soccer league code (e.g., 'eng.1', 'por.1')
Returns:
Logo downloader league key (e.g., 'soccer_eng.1', 'soccer_por.1')
"""
return f"soccer_{league_code}"
# Convenience function for easy integration
def download_missing_logo(league: str, team_id: str, team_abbreviation: str, logo_path: Path, logo_url: str | None = None, create_placeholder: bool = True) -> bool:
"""
Convenience function to download a missing team logo.
Args:
team_abbreviation: Team abbreviation (e.g., 'UGA', 'BAMA', 'TA&M')
league: League identifier (e.g., 'ncaa_fb', 'nfl')
logo_path: Full path to where the logo should be saved
logo_url: Optional direct URL to the logo
create_placeholder: Whether to create a placeholder if download fails
Returns:
True if logo exists or was successfully downloaded, False otherwise
"""
downloader = LogoDownloader()
# Use the directory from the logo_path parameter (respects config settings)
logo_path = Path(logo_path)
if not logo_path.is_absolute():
project_root = Path(__file__).resolve().parents[1]
logo_path = (project_root / logo_path).resolve()
logo_dir = str(logo_path.parent)
# Ensure the directory exists and is writable
if not downloader.ensure_logo_directory(logo_dir):
logger.error(f"Cannot download logo for {team_abbreviation}: directory {logo_dir} is not writable")
return False
# Use the exact filepath that was passed in (respects config settings)
filepath = logo_path
if filepath.exists():
logger.debug(f"Logo already exists for {team_abbreviation} ({league})")
return True
# Try to download the real logo first
logger.info(f"Attempting to download logo for {team_abbreviation} from {league}")
if logo_url:
success = downloader.download_logo(logo_url, filepath, team_abbreviation)
if success:
time.sleep(0.1) # Small delay
if not success and create_placeholder:
logger.info(f"Creating placeholder logo for {team_abbreviation}")
success = downloader.create_placeholder_logo(team_abbreviation, logo_dir)
return success
success = downloader.download_missing_logo_for_team(league, team_id, team_abbreviation, logo_path)
if not success and create_placeholder:
logger.info(f"Creating placeholder logo for {team_abbreviation}")
# Create placeholder as fallback
success = downloader.create_placeholder_logo(team_abbreviation, logo_dir)
if success:
logger.info(f"Successfully handled logo for {team_abbreviation}")
else:
logger.warning(f"Failed to download or create logo for {team_abbreviation}")
return success
def download_all_logos_for_league(league: str, force_download: bool = False) -> Tuple[int, int]:
"""
Convenience function to download all missing logos for a league.
Args:
league: League identifier (e.g., 'ncaa_fb', 'nfl')
force_download: Whether to re-download existing logos
Returns:
Tuple of (downloaded_count, failed_count)
"""
downloader = LogoDownloader()
return downloader.download_missing_logos_for_league(league, force_download)