Compare commits

..
3 Commits
Author SHA1 Message Date
ChuckandClaude Opus 5.5 adb4520f3d fix: call forget_manager_fonts through a hasattr check pylint can follow
getattr(..., None) then callable() is fine at runtime, but pylint's E1102
("not callable") can't see through it, and Codacy fails the check on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 13:29:50 -04:00
Chuck 93123aae7a Merge branch 'main' of https://github.com/ChuckBuilds/LEDMatrix into claude/fonts-used-by 2026-09-23 13:23:14 -04:00
ChuckandClaude Opus 5.5 d3d49858de feat(web): show which plugins use each font, warn before deleting one
The Fonts tab lists font files from the web process's own scan, and the
plugins that register fonts run in the display process, so the tab had no
way to say whether a font was in use before deleting it.

The display service now publishes {catalog key: [plugin ids]} to the
shared cache (font_usage_snapshot, src/font_usage.py), built from the
loaded plugins' FontManager.register_manager_font() registrations. A
daemon thread checks every 10 s and writes only when the usage changed
(plus a daily refresh so cache cleanup cannot expire it); it never raises.
Families, aliases (press_start, four_by_six, ...) and paths are resolved
through FontManager's catalog to the file stem the Fonts tab keys rows by;
fonts outside assets/fonts are left out. Unloading a plugin drops its
registrations (new FontManager.forget_manager_fonts).

GET /api/v3/fonts/catalog merges used_by into each row per request (the
5-minute scan cache is copied, never edited): a list of plugin ids, or
null when the display service has not reported. The tab shows a Used by
column ("unknown" / "-" / ids, rendered as text) and deleting an in-use
font names the plugins in the confirmation, from a fresh read. The server
still refuses only system fonts. Catalog fetches bypass the browser's
5-second API cache, which otherwise served the pre-delete list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 13:11:51 -04:00
462 changed files with 32549 additions and 35785 deletions
+1
View File
@@ -4,3 +4,4 @@ exclude_paths:
- "plugins/**"
- "assets/**"
- "test/**"
- "scripts/debug/**"
-6
View File
@@ -1,8 +1,2 @@
# Auto detect text files and perform LF normalization
* text=auto
# Files the Pi executes must stay LF even in a Windows checkout with
# core.autocrlf=true: a CRLF shebang line fails with "bad interpreter",
# and systemd rejects CRLF unit files.
*.sh text eol=lf
*.service text eol=lf
+16 -6
View File
@@ -3,13 +3,21 @@ name: Claude Code Review
on:
pull_request:
types: [opened, synchronize, ready_for_review, reopened]
# Optional: Only run on specific file changes
# paths:
# - "src/**/*.ts"
# - "src/**/*.tsx"
# - "src/**/*.js"
# - "src/**/*.jsx"
jobs:
claude-review:
# Pull requests from forks get no repository secrets, so without this
# guard every outside contributor's PR showed this check red for a reason
# they can't fix. Skipped checks don't block merging.
if: github.event.pull_request.head.repo.full_name == github.repository
# Optional: Filter by PR author
# if: |
# github.event.pull_request.user.login == 'external-contributor' ||
# github.event.pull_request.user.login == 'new-developer' ||
# github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR'
runs-on: ubuntu-latest
permissions:
contents: read
@@ -19,13 +27,13 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code Review
id: claude-review
uses: anthropics/claude-code-action@756cc22e19660d20e8cc9496b4f242475a7f7790 # v1
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# Review PRs opened by the Claude GitHub App. Without this the action
@@ -37,4 +45,6 @@ jobs:
plugin_marketplaces: 'https://github.com/anthropics/claude-code.git'
plugins: 'code-review@claude-code-plugins'
prompt: '/code-review:code-review ${{ github.repository }}/pull/${{ github.event.pull_request.number }}'
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://code.claude.com/docs/en/cli-reference for available options
+10 -2
View File
@@ -26,13 +26,13 @@ jobs:
actions: read # Required for Claude to read CI results on PRs
steps:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code
id: claude
uses: anthropics/claude-code-action@756cc22e19660d20e8cc9496b4f242475a7f7790 # v1
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
@@ -40,3 +40,11 @@ jobs:
additional_permissions: |
actions: read
# Optional: Give a custom prompt to Claude. If this is not specified, Claude will perform the instructions specified in the comment that tagged it.
# prompt: 'Update the pull request description to include a summary of changes.'
# Optional: Add claude_args to customize behavior and configuration
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://code.claude.com/docs/en/cli-reference for available options
# claude_args: '--allowed-tools Bash(gh pr *)'
+3 -70
View File
@@ -8,7 +8,7 @@ on:
# needs a re-run or didn't get created.
workflow_dispatch:
# The jobs only check out the repo and run the tests.
# Both jobs only check out the repo and run pytest.
permissions:
contents: read
@@ -35,7 +35,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt -r web_interface/requirements.txt -r requirements-test.txt
pip install -r requirements.txt -r requirements-test.txt
pip install RGBMatrixEmulator
- name: Run plugin safety harness
@@ -58,7 +58,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt -r web_interface/requirements.txt -r requirements-test.txt
pip install -r requirements.txt -r requirements-test.txt
pip install RGBMatrixEmulator
# Run the ENTIRE test tree (except test/plugins, which the
@@ -73,70 +73,3 @@ jobs:
--cov=src --cov=web_interface \
--cov-report=term \
--cov-fail-under=52
js-tests:
name: Web UI JS tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
python-version: "3.12"
cache: pip
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: "22"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt -r web_interface/requirements.txt
npm install --no-audit --no-fund --prefix test/js
# The DOM suites test the real server-rendered pages and API, so they
# need the web interface running. REQUIRE_DOM turns "couldn't reach it"
# into a failure instead of a silent skip.
- name: Start the web interface
run: |
EMULATOR=true python -c "from web_interface.app import app; app.run(host='127.0.0.1', port=5000, threaded=True)" > web.log 2>&1 &
for i in $(seq 60); do curl -sf -o /dev/null http://127.0.0.1:5000/ && exit 0; sleep 1; done
cat web.log
exit 1
- name: Run JS suites
env:
BASE: http://127.0.0.1:5000
REQUIRE_DOM: "1"
run: node test/js/run_all.js
type-check:
name: Type check (mypy ratchet)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
python-version: "3.12"
cache: pip
# The runtime requirements are installed so mypy sees the real types of
# PIL, requests, psutil and friends -- missing, they'd be Any and the
# result would differ from a developer's machine. mypy and the stubs are
# pinned so a new release can't turn this red without a code change.
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt -r web_interface/requirements.txt
pip install mypy==1.20.2 types-requests==2.33.0.20260906 types-pytz==2026.4.0.20260926
# mypy on exactly the modules in mypy-clean.txt; fails on any error in
# them, or if a listed file is missing. See CONTRIBUTING.md.
- name: Run mypy on the ratchet list
run: python scripts/check_types.py
+14 -11
View File
@@ -3,13 +3,15 @@ __pycache__/
*.py[cod]
*$py.class
# Secrets and per-device state. Everything the software writes into config/
# is local to one device -- config.json, config_secrets.json, wifi_config.json,
# ytm_auth.json (a login session), saved_repositories.json, font_overrides.json,
# and the temp files atomic writes leave behind when interrupted -- so only
# the templates are tracked. Listing files one by one missed several.
config/*
!config/*.template.json
# Secrets
config/config_secrets.json
# Atomic writes leave these behind when a save or a test is interrupted;
# the suite drops several per run.
config/.config_secrets.json.tmp.*
config/config.json
config/config.json.backup
config/wifi_config.json
config/uninstalled_plugins.json
credentials.json
token.pickle
@@ -50,6 +52,7 @@ config/backups/
# Starlark apps runtime storage (installed .star files and cached renders)
/starlark-apps/
skin_renders/
# JS test deps (test/js)
node_modules/
@@ -79,10 +82,10 @@ assets/stocks/crypto_icons/
# Plugin operation state written at runtime.
#
# web_interface/app.py writes data/plugin_state.json and data/operation_history.json
# (older releases also data/plugin_operations.json) as the web interface runs, into
# a directory that ships tracked (data/.gitkeep). Unignored, every rig that ever
# opened the web UI -- and every test run that constructs the app -- would leave
# web_interface/app.py writes data/plugin_operations.json, data/plugin_state.json
# and data/operation_history.json as the web interface runs, into a directory that
# ships tracked (data/.gitkeep) and was otherwise unignored. So every rig that ever
# opened the web UI -- and every test run that constructs the app -- left three
# untracked files behind and a permanently dirty `git status`. Same reasoning as
# the logo rule above: a checkout that is always dirty is a checkout nobody reads.
data/*
+5 -13
View File
@@ -37,22 +37,14 @@ repos:
types: [python]
pass_filenames: false
# The mypy ratchet -- the same check as CI's "Type check (mypy ratchet)"
# job: mypy on exactly the modules listed in mypy-clean.txt. Run it with
# pre-commit run mypy --hook-stage manual
# A local hook rather than mirrors-mypy so mypy sees the packages installed
# from requirements.txt, as CI does; an isolated hook env without them types
# PIL, requests and friends as Any and reports different errors. Needs
# mypy==1.20.2 (the version CI pins) in the environment you commit from.
- repo: local
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.8.0
hooks:
- id: mypy
name: mypy (ratchet, mypy-clean.txt)
entry: python scripts/check_types.py
language: system
additional_dependencies: [types-requests, types-pytz]
args: [--ignore-missing-imports, --no-error-summary]
pass_filenames: false
always_run: true
stages: [manual]
files: ^src/
- repo: https://github.com/PyCQA/bandit
rev: 1.8.3
+83 -553
View File
@@ -19,121 +19,18 @@ accepts both, but the store flags the old spelling as deprecated
## Unreleased
## 3.6.2
A fix to `src.common.favorite_team_check` (#670).
### Fixes
- The favourite-team check no longer says the Europa League season has
finished between matchdays. Its scoreboard keeps showing the last matchday,
and its calendar is a "list" of rounds rather than match days, so neither
3.6.1 rule applied. When every event is past, a round in a list calendar
that has not started yet (outside an offseason phase) now draws no
conclusion. PLL, the World Cup and AFL, whose seasons are over, are still
reported as finished: no round of theirs is still to start. (#670)
## 3.6.1
A fix to `src.common.favorite_team_check` (#667). Plugins that drop their
bundled copy of it should floor on 3.6.1, not 3.6.0.
### Fixes
- The favourite-team check no longer logs "the season has finished" for a
league that is still playing. ESPN's default scoreboard keeps showing the
last slate after it: MLB's regular-season games two days into the
postseason, a soccer league's previous matchday between rounds. When every
event is in the past, the check now looks first at the league's phase (a
regular season or postseason that has moved past the events shown draws no
conclusion) and at a match-day calendar (`calendarType` "day" with
`calendarIsWhitelist`, as soccer, the NHL and the NBA use), whose next date
becomes "nothing on until <date>". An offseason, or a payload without these
fields, is reported as before.
## 3.6.0
New modules a plugin may import via `src.*` (floor on 3.6.0). Both are
promoted from files the scoreboard plugins carry as copies; the plugins keep
their copies as a fallback until they floor on 3.6.0. No other change since
3.5.0.
- `src/common/favorite_team_check.py` — `FavoriteTeamCheck(logger, leagues)`:
checks configured favourite team codes against ESPN once per league, on a
daemon thread, and logs why a league shows nothing (a wrong code, with the
nearest real one, or a season that has not started). The seven copies
(`<sport>_favorite_check.py`) were byte-identical; this is the same code,
with type annotations added.
- `src/common/sports_timezone.py` — `resolve_timezone_name()` /
`resolve_timezone()` (plus `system_timezone_name()`): the timezone a
scoreboard draws start times in. The ten copies (`<sport>_timezone.py`)
differed only in two values, which are keyword-only arguments here:
`plugin_label` (named in the warning logged when nothing resolves) and
`writeback_fixed_in` (for a plugin that once wrote `"UTC"` back into the
saved config; `None` otherwise). Same resolution order and log messages.
## 3.5.0
New modules a plugin may import via `src.*` (floor on 3.5.0):
- `src/common/sports_helpers.py` — the helpers the scoreboards' `sports.py`
carry byte-identical copies of: `clamp_window`, `clamp_seconds`,
`logo_needs_refresh`, `spread_weighted_order` (+ `MIN_WINDOW_DAYS`,
`MAX_WINDOW_DAYS`), and `SportsHelpersMixin` with `_mode_customization`,
`_setting_int`, `_reset_dwell_on_reentry`, `_next_switch_index`,
`_spread_weighted_order`, `_odds_color`, `_upcoming_date_and_time_text` under
the plugins' names and signatures, plus the `_favorite_key` override point.
Constructor-free; keeps lazy state on its host (see the module docstring,
which also gives the host contract).
A new module rather than more methods on `sports_shared`: a plugin that
deletes a copy and leans on an older module having grown the method fails at
runtime with `AttributeError`, which no load-time check sees, while a missing
module fails at load. Nothing in core uses it yet.
- `src/common/espn_dates.py` — `fetch_espn_scoreboard`,
`fetch_espn_date_chunks`, `espn_date_chunks`, `clamp_espn_limit`,
`ESPN_MAX_LIMIT`: fetch an ESPN scoreboard date range now that ESPN rejects
ranges (see Sports data below). Plugins bundle a copy of it.
- `src/common/json_body.py` — `response_json(response)`: `response.json()`,
parsed by orjson when it is installed (an optional dependency) and by the
stdlib otherwise; an orjson parse error falls back to `response.json()` so
requests raises its usual error. Same Python objects either way; a season
schedule parses about 1.7x faster on a Pi 4, and the parse holds the GIL (so
freezes the display) for that much less time. `espn_dates` and
`BackgroundDataService` use it; `espn_dates` falls back to `response.json()`
when it is missing, so the plugins' bundled copies of `espn_dates` still load
on an older core.
- `src/common/bdf_font.py` — `load_bdf_face(path, size)` (a cached
`freetype.Face` plus the pixel size it really renders at, falling back to
the file's native strike) and `draw_bdf_text(draw, text, x, y, face, color)`.
`DisplayManager`, `FontManager`, `element_style` and the plugin test harness
now all load and draw BDF text through it; the panel's pixels are unchanged
and BDF text draws 10-250x faster. The plugin test harness's
`calendar_font` / `bdf_5x7_font` now has the panel's 7px size set: it used
to be an unsized face, so in golden images and `check_plugin` /
`dev_server` previews its text sat 6px above where the panel draws it (off
the canvas entirely near the top) and `get_font_height()` returned 0.
Also new under `src/` since 3.4.0, but internal to core rather than for plugins:
`src/common/frame_timing.py` and `src/common/render_gate.py` (see Scrolling),
`src/core_config_keys.py`, `src/deprecation.py`, `src/device_location.py`,
`src/font_usage.py`, `src/matrix_support.py`, `src/pi5_matrix_support.py`,
`src/redaction.py`, `src/scan_order.py`, `src/web_interface/config_arrays.py`,
and in `src/plugin_system/`: `plugin_dirs.py`, `repo_urls.py`,
`store_install.py`, `store_registry.py` and `store_update.py`.
New names in existing modules (a plugin using these must floor on 3.5.0):
- `src.common.api_helper`: `USER_AGENT`, `DEFAULT_HTTP_HEADERS` (read-only).
- `src.logo_downloader`: `fetch_logo`, `save_png_atomically`,
`shared_downloader`.
- `src.common.sports_card.unshare_element_fonts` takes an optional third
argument, `element_for_font` (default: the module's `ELEMENT_FOR_FONT`, so
existing calls are unchanged).
- `FontManager.get_font()` returns a BDF font at its native size when asked for
a size the file doesn't contain (5x7.bdf at 8 or 10px, say). It used to
return PIL's default font, a different typeface, so a plugin that relied on
that will now render the font it asked for.
- `src.wifi_manager.get_wifi_status_path()` — where WiFi status messages for
the display are written (`config/wifi_status.json`).
- `BackgroundDataService.handles_espn_date_ranges` (see Sports data).
- `FontManager.register_plugin_fonts()` takes an optional `plugin_dir`, and
`FontManager.forget_manager_fonts()` is new (see Fonts).
- The web UI's Fonts tab has a **Used by** column: the loaded plugins that
registered each font with `FontManager.register_manager_font()`, published
by the display service to the shared cache (`src/font_usage.py`) and merged
into `GET /api/v3/fonts/catalog` as `used_by`. Deleting a font a plugin
uses now names those plugins in the confirmation (it is not blocked).
`FontManager.forget_manager_fonts()` is new; unloading a plugin calls it.
Deprecated, removed in 3.7.0 (each logs a warning on first use; see
`docs/PLUGIN_API_REFERENCE.md#deprecated-apis` for replacements). Nothing in
@@ -153,6 +50,76 @@ core, the monorepo or the registry's third-party plugins calls them:
`unregister_plugin_fonts`.
- `PluginManager.get_enabled_plugins`.
### Config writes
- A power cut or crash mid-save can no longer leave `config/config.json`
truncated. `ConfigManager.save_config()` wrote the file in place; it,
`save_config_atomic()`, `save_raw_file_content()` and backup rollback now
share one writer (`atomic_write_text` in `src/config_manager_atomic.py`)
that fsyncs a temp file, renames it into place and fsyncs the directory.
- `save_config_atomic()` no longer rewrites `config_secrets.json` on every
save, only when its content changes, and rotating backups no longer re-reads
every backup. The backups themselves are unchanged:
`config/backups/config.json.backup.<version>` plus its paired secrets
backup, five newest kept.
- A save by the root-run display service keeps the file's previous owner
instead of handing `config.json` to root, and an install path with
"secrets" in a directory name no longer makes `config.json` mode 0640.
New names in existing modules (no new modules; a plugin importing these must
floor on the release that ships them):
- `src.common.api_helper`: `USER_AGENT`, `DEFAULT_HTTP_HEADERS` (read-only).
- `src.logo_downloader`: `fetch_logo`, `save_png_atomically`,
`shared_downloader`.
### Logo downloads
- `download_missing_logo` / `LogoDownloader.download_logo` (the path the
scoreboard plugins use) now stream the logo with a 10 MB cap, accept only an
`image/*` response that Pillow can decode, and move the finished RGBA PNG
into place atomically. A failed, oversized or non-image download no longer
leaves a partial file behind, and no longer replaces a logo already on disk.
`LogoHelper._download_logo` goes through the same code. Signatures and return
values are unchanged; saved files are pixel-identical to before.
- `download_missing_logo` reuses one downloader (one `requests.Session`) per
thread instead of building a new one for every logo.
- Placeholder logos are written atomically, without the `test_write.tmp`
probe file.
### HTTP headers
- The logo downloader and the background data service send the real
`LEDMatrix/1.0 (+https://github.com/ChuckBuilds/LEDMatrix)` User-Agent
instead of a `yourusername` / `contact@example.com` placeholder, and no
longer set `Accept-Encoding: ... br` by hand (brotli is not installed, so a
`br` response could not be decoded); requests picks the encodings.
## 3.5.0
New modules a plugin may import via `src.*` (floor on 3.5.0):
- `src/common/sports_helpers.py` — the helpers the scoreboards' `sports.py`
carry byte-identical copies of: `clamp_window`, `clamp_seconds`,
`logo_needs_refresh`, `spread_weighted_order` (+ `MIN_WINDOW_DAYS`,
`MAX_WINDOW_DAYS`), and `SportsHelpersMixin` with `_mode_customization`,
`_setting_int`, `_reset_dwell_on_reentry`, `_next_switch_index`,
`_spread_weighted_order`, `_odds_color`, `_upcoming_date_and_time_text` under
the plugins' names and signatures, plus the `_favorite_key` override point.
Constructor-free; keeps lazy state on its host (see the module docstring,
which also gives the host contract).
A new module rather than more methods on `sports_shared`: a plugin that
deletes a copy and leans on an older module having grown the method fails at
runtime with `AttributeError`, which no load-time check sees, while a missing
module fails at load. Nothing in core uses it yet.
- `test/test_common_is_hardware_free.py` — `src/common` must import without
`rgbmatrix` and never import `src.base_classes`, `src.display_manager` or
`src.plugin_system` at module level.
- `src/common/espn_dates.py` — `fetch_espn_scoreboard`,
`fetch_espn_date_chunks`, `espn_date_chunks`, `clamp_espn_limit`,
`ESPN_MAX_LIMIT`: fetch an ESPN scoreboard date range now that ESPN rejects
ranges (see Sports data below). Plugins bundle a copy of it.
### Config saves and plugin config preparation
- A JSON `POST /api/v3/config/main` changes only the keys it sends. The MQTT
@@ -198,23 +165,8 @@ core, the monorepo or the registry's third-party plugins calls them:
"enabled but not found in plugins directory", and plugin ids that collide
with any core config section are flagged: the last private copies of the
core-key list now use `src/core_config_keys.py`.
- Plugin config saves recombine position-keyed inputs for nullable array fields (`"type": ["array", "null"]`).
- A blank Max Dynamic Duration keeps the stored value instead of failing the Display save with a 500; other values must be whole seconds from 30 to 1800.
- A power cut or crash mid-save can no longer leave `config/config.json`
truncated. `ConfigManager.save_config()` wrote the file in place; it,
`save_config_atomic()`, `save_raw_file_content()` and backup rollback now
share one writer (`atomic_write_text` in `src/config_manager_atomic.py`)
that fsyncs a temp file, renames it into place and fsyncs the directory.
- `save_config_atomic()` no longer rewrites `config_secrets.json` on every
save, only when its content changes, and rotating backups no longer re-reads
every backup. The backups themselves are unchanged:
`config/backups/config.json.backup.<version>` plus its paired secrets
backup, five newest kept.
- A save by the root-run display service keeps the file's previous owner
instead of handing `config.json` to root, and an install path with
"secrets" in a directory name no longer makes `config.json` mode 0640.
### Sports data, logos and odds
### Sports data
- Since 2026-09-15 ESPN answers `dates=YYYYMMDD-YYYYMMDD` scoreboard queries
with `400 Bad Request` for every sport, so season schedules, the weeks window
@@ -262,44 +214,6 @@ core, the monorepo or the registry's third-party plugins calls them:
longer swallowed as a missing poll. This is the implementation the football,
baseball and hockey boards already ship; core was the last copy on the old
one.
- `BaseOddsManager.get_odds()` no longer returns the cached "no odds" marker (`{"no_odds": True}`) as if it were odds. A game ESPN had no odds for is cached that way so it isn't re-requested every update; on the next update the cache hit handed the marker back, and callers saw a truthy dict. It now returns `None` for it, on the cache hit and in the stale-cache fallback after a failed fetch, as the plugins' bundled copies already did.
- Background data fetches retry at one level instead of two. The session adapter retried a connection error three times inside every attempt of the service's own retry loop, so a dead network cost up to 16 connection attempts per request and held one of the few worker threads throughout; now it is the loop's `max_retries + 1` attempts. ESPN date-range chunks, which don't go through that loop and skip a chunk that fails, keep a small connection retry of their own so a brief blip doesn't drop a month from a cached season.
- `LogoHelper.load_logo_with_download()` sizes its placeholder to the scaled logo box, like a real logo (only differs when `scale` isn't 1).
- The AP Top 25 resolver remembers a failed or empty rankings fetch for 5 minutes, so an ESPN outage no longer costs every scoreboard update a 30s timeout. Its duplicate INFO log line is gone.
- `BackgroundDataService` runs a cache-hit callback outside its lock, as the fetch path does.
- `LogoHelper.load_logo_with_download()` waits an hour before retrying a download that failed for a missing logo, instead of retrying (with a 30 s timeout) on every call.
- Restamping a placeholder logo writes the file atomically.
- The odds manager logs cache hits, misses and fetches at DEBUG, and a bad JSON body is logged as a parse error rather than a failed fetch.
- `APIHelper` keeps cached responses for the `cache_ttl` it was given, instead of always 300 s.
- Logo scales from 0.1 to 10 are honoured everywhere; values outside that range are clamped.
- `LogoHelper` and `logo_downloader`: an empty ESPN logo list counts as a failed download, and the placeholder is written at the requested path.
- The `SportsCoreSharedMixin` helpers that behave identically to their
`sports_card` twins (`_card_option`, `_vs_text`, `_format_game_time`,
`_coerce_rgb`, `_crisp_size`, `_unshare_element_fonts`, the colour/month/
weekday/font-grid tables) are now thin wrappers over the `sports_card`
functions, and `_format_game_date` / `_schema_font_size` share its
formatting body and schema parser. No method was removed or renamed and
nothing renders differently: `test/test_sports_twins.py` checks each pair
against the same inputs, and the old and new mixin agree on every input
there. The pairs that do differ -- favourite-result colours on nested
payloads, the weekday's timezone, the element-name map, per-mode colours --
are left as they are and pinned in that test.
- `download_missing_logo` / `LogoDownloader.download_logo` (the path the
scoreboard plugins use) now stream the logo with a 10 MB cap, accept only an
`image/*` response that Pillow can decode, and move the finished RGBA PNG
into place atomically. A failed, oversized or non-image download no longer
leaves a partial file behind, and no longer replaces a logo already on disk.
`LogoHelper._download_logo` goes through the same code. Signatures and return
values are unchanged; saved files are pixel-identical to before.
- `download_missing_logo` reuses one downloader (one `requests.Session`) per
thread instead of building a new one for every logo.
- Placeholder logos are written atomically, without the `test_write.tmp`
probe file.
- The logo downloader and the background data service send the real
`LEDMatrix/1.0 (+https://github.com/ChuckBuilds/LEDMatrix)` User-Agent
instead of a `yourusername` / `contact@example.com` placeholder, and no
longer set `Accept-Encoding: ... br` by hand (brotli is not installed, so a
`br` response could not be decoded); requests picks the encodings.
### Scrolling
@@ -330,92 +244,6 @@ core, the monorepo or the registry's third-party plugins calls them:
held 20 ms frame as missed refreshes, and Vegas `frame_based_scrolling` /
`scroll_delay` are described as the speed clamp they are rather than frame
stepping. Scoreboard `scroll_delay` is documented as ignored for pacing.
- `ScrollHelper.set_scrolling_image()` accepts RGBA, L and palette images (transparent pixels become black), and a new scrolling image no longer jumps ahead by the time the helper sat idle.
- `src.common.frame_timing` -- times every frame the display presents, whoever
drew it, and writes cumulative counters to `/dev/shm`. Two tools read it:
`scripts/frame_soak.py` judges a running service (late frames, freezes,
where the time goes), and `scripts/render_bench.py` judges the hardware and
render path alone on a synthetic strip. Both fail a run above 0.1% late
frames, and both call a loop that never waited for the panel NOT LOCKED. A
stall watchdog logs the stack of whatever holds a scroll up for 250 ms or
more. See `docs/SCROLL_PERFORMANCE.md`, "Soaking a rig".
- `display.scan_order_compensation` (`"auto"` by default): while something
scrolls at one pixel per refresh, one half of each panel is shown a refresh
behind the other, which removes the 1px step a 1:N-scan panel shows across
its middle. Only for layouts whose row order is known; `"off"` disables it.
See `docs/SCROLL_PERFORMANCE.md`, "A tear across the middle on fast scrolls".
### Display and Vegas
- Vegas scrolls in step with the panel's refresh (#628). With `smooth_scroll`
(on by default) the strip moves a whole number of pixels per presented
frame, each held for `frame_hold` refreshes and timed by `SwapOnVSync`,
the same pacing as the plugin tickers; it used to advance by elapsed time
and sleep to `target_fps`, missing a vsync every few frames. The speed is
solved against the panel's measured refresh when that is below its
`limit_refresh_rate_hz` cap. The old sub-pixel blend, which the panel shows
as shimmer, is kept as `vegas_scroll.sub_pixel_blend` (default off). While
scrolling, the web preview's PNG is encoded on a background writer instead
of the render thread. On a Pi 4 driving 512x64, late frames went from about
6.4% to 0.7%.
- Vegas prepares plugin content off the render thread (#630). A plugin that
needed the shared canvas used to be fetched on the render thread, stalling
the scroll for as long as it took (320 ms for news, 660 ms for a hockey
scoreboard, measured). `DisplayManager.offscreen(width, height)` gives the
calling thread a canvas of its own: `image`, `draw` and `matrix` are now
properties that resolve to it inside the block, where `update_display()`,
the hardware half of `clear()` and `set_scrolling_state()` /
`set_frame_hold()` do nothing. Background fetches take the plugin's lock,
waiting up to 2 s for a running `update()` and otherwise skipping the plugin
that round. A GIL gate (`src/common/render_gate.py`) pauses the prefetch
thread outside a window around each vsync swap, so the render thread finds
the GIL free; it needs the rebuilt binding that releases the GIL in
`SwapOnVSync` and stays off (one INFO line per Vegas run) on a stock one.
New `vegas_scroll` keys: `offscreen_prefetch` and `prefetch_gate` (both on by
default) and `switch_interval_ms` (experimental, default 0, off). Design in
`docs/OFFSCREEN_RENDERING.md`.
- On-demand requests, the display on/off schedule and brightness take effect
within about a quarter of a second instead of at the next screen (#618). A
screen can stay up for a minute and a Vegas iteration for 240 s, so an
on-demand request during Vegas waited for the iteration to end and a
brightness save mid-screen could be lost. Vegas now stops for an on-demand
request and for the display being scheduled off, and a brightness change
re-sends the current frame. Plugin enable/disable, screen durations and Vegas
settings still apply at the next screen.
- The Rotation & Durations page takes effect (#605). A saved
`display.display_durations` value now wins over the plugin's own duration;
the plugin was asked first, and every plugin inherits
`get_display_duration()`, so saved values did nothing. The page shows an
unsaved screen blank with the plugin's own duration as the placeholder (it
showed 30 where the real default is 15), and saving a blank removes the
override. Durations saved before this now apply.
- Vegas settings reach a running scroll (#605): they are queued when
`display.vegas_scroll` changes (unrelated saves don't rebuild the strip) and
also applied while Vegas is stopped. The sync follower's scroll-speed
default (75) now matches `VegasModeConfig`'s (50). The Vegas live-priority
scan is throttled to 4 Hz.
- `DisplayManager.defer_update()` from a plugin's update thread no longer loses queued updates while the render thread processes the queue; the queue is locked, and the queued callables still run outside the lock.
- **Behaviour change:** when `display.hardware.limit_refresh_rate_hz` is missing from config, the panel is now capped at 100 Hz (the config template's value) instead of 90 Hz. Scroll pacing already assumed 100 Hz in that case, so it now matches what the panel does. Configs that set the key (every config migrated from the template) are unaffected.
- A sync follower adopts the leader's scroll image between frames on the render thread, instead of the TCP thread swapping the image, array and width while a frame is being drawn.
- `update_display()` errors are logged once with a traceback, then at most once a minute with a count, instead of an untraced line every frame. Several swallowed exceptions in `DisplayController` now log at DEBUG.
- The repo-root `display_controller.py` now runs `run.py` (the real entry point), so it gets run.py's `-e`/`-d` flags, logging setup and `sys.dont_write_bytecode`.
- Vegas: a plugin set to `vegas_mode: "static"` pauses the scroll for its turn again. The pause was triggered by peeking at the front of a segment buffer that continuous scrolling (the default) never advances, so a static plugin paused only if it happened to be first, once, at startup, and otherwise scrolled past as ordinary content; swap mode had the same problem for any static plugin not first in its cycle. The render pipeline now marks where each static plugin's turn falls in the strip and the scroll pauses when it gets there. The pause runs the plugin's `display()` under its plugin lock, and a static plugin's content is no longer rendered for the strip.
- The display loop no longer spins at 100% CPU when no enabled mode has anything to show (for example, only a sports plugin enabled in its off-season). After one full rotation of empty modes it checks one mode per second until something shows; live content still takes over at once.
- Stopping `ledmatrix.service` runs the controller's cleanup (SIGTERM now takes the Ctrl-C path).
- Turning Vegas on in the web UI works without a restart when it was off at startup.
- Vegas comes back after live content interrupts it. It stayed paused, and the display fell back to normal rotation until a restart.
- A day with dimming turned off in a per-day dim schedule stays at normal brightness. Before, brightness went back to dim for most of each minute.
- Stopping on-demand after a second request resumes rotation where it was first interrupted, not at the first request's screen.
- Turning Vegas off and on no longer shows content prepared for the previous run, including plugins disabled in between.
- How long a Vegas iteration runs is timed with the monotonic clock, so an NTP clock step on a Pi without an RTC doesn't cut it short or stretch it.
- The sync status file is removed when the display service stops, and at startup in standalone mode, so the web UI no longer reports a peer from an earlier run. Concurrent writes each use their own temp file.
- `render_gate.swap_releases_gil()` delegates to `frame_timing.binding_releases_gil()` instead of duplicating it.
- Vegas `max_cycle_duration` defaults to 240 s when unset, as documented (it was 600 s). The Vegas defaults are now defined once.
- The display controller stops Vegas mode on shutdown.
- Startup validation warnings are logged once, not twice.
- Vegas logs one INFO line per plugin-list refresh.
- `run.py -d` shows `display_manager` debug output.
- Removed: the Vegas staging buffer that was never filled (`swap_buffers()`, and `staging_count` / `current_index` in `get_buffer_status()`), unread `ContentSegment` fields, and `geometry.find_blank_cut()`.
### Web interface
@@ -480,142 +308,8 @@ core, the monorepo or the registry's third-party plugins calls them:
discover when nothing has been discovered yet, and rescan once when a
specific plugin id (or, for on-demand by mode, a mode) is not found, so a
plugin installed since the last scan is found too.
- `web_interface/blueprints/api_v3/plugins.py` (3,285 lines) is split by area into `plugins.py` (installed list, enable/disable, plugin actions), `plugin_store.py`, `plugin_config.py`, `plugin_assets.py`, `plugin_health.py`, `plugin_operations.py` and `plugin_calendar.py`. Pure move: every function body and route decorator is byte-identical, and URLs and endpoint names are unchanged.
- Plugins installed as `ledmatrix-<id>` (or in a directory not named after their id) work in the installed list, the update button, recorded versions, the plugin config form and plugin web UI pages. Those routes built `plugins_dir/<id>` themselves instead of asking the plugin manager.
- Uploading several plugin images checks every file before saving any, so a rejected file no longer leaves the others saved; the images' `.metadata.json` and the calendar plugin's `credentials.json` are written atomically, and the credentials upload no longer returns the server's absolute path.
- `"false"` sent as a string no longer counts as true when toggling a plugin (including Starlark apps) or starting on-demand mode (`pinned`, `start_service`); `force` on the AP-enable route is parsed like every other WiFi boolean (`"yes"` and `1` now force).
- The live-preview stream starts a new broadcast thread for a client that connects while the previous one is shutting down; that client got no updates.
- The web server's log filter no longer raises when werkzeug logs with `exc_info=True`.
- The raw secrets editor's save errors carry `error_code` like the main config's; the asset delete route answers 400 for a missing body instead of 415/500. Dead code removed: an unused manifest scan on each Plugins-tab load, backup routes' duplicate catch-alls, redundant imports.
- A plugin's own config widget (`/static/plugin-widgets/<id>/<widget>.js`) is requested with `?v=<plugin version>`, so an updated plugin's widget reaches browsers instead of the copy cached as immutable for a year.
- A failed installed-plugins reload after a toggle, install or uninstall shows one error, not a second generic "unexpected error" toast.
- The timezone picker on the General tab renders again when the tab is reloaded in the same page session.
- Removed dead code: the plugin-action button's six plugin-id fallbacks (the button always passes its id) and its `[DEBUG]` logging, `window.currentPluginConfig` (never set to anything but `null`), the file-upload widget's JSON delete branch (its endpoint never existed), unused `PluginAPI` / `PluginInstallManager` / `PluginStateManager` helpers, `loadPluginWidgetsFromManifest`, no-longer-reachable fallbacks for a stale `install_manager.js` and a missing `LEDVisibility`, and 13 unused CSS utility rules.
- The Logs tab's "Now showing" no longer reads "unknown" when one screen stays up longer than 2 minutes.
- A network failure fetching GitHub repo info logs a warning, not an error.
- The Operation History plugin filter lists installed plugins (it showed one option, "plugins").
- Ctrl/Cmd+S submits the active tab's visible form (with its validation) instead of the first form in the page; it does nothing inside a dialog or on a tab without a form. The Ctrl/Cmd+R override (the browser's own reload) and the textarea auto-resize (no textarea exists at load) are removed.
- Tools tab actions and diagnostics show the server's error message; only a non-JSON error falls back to `HTTP <status>`.
- An uninstalled plugin no longer reappears in the installed list: writes through `PluginAPI` clear its 5s GET cache, and Refresh and the post-uninstall reload bypass both list caches.
- Plugin widgets load from `/static/plugin-widgets/` only; the two other paths it tried have no route.
- The raw JSON editor escapes the parse error, and the slider widget escapes its value, min, max and step.
- Removed unused array-of-objects and key-value helpers from `plugins_manager.js` (about 640 lines, no callers) and a redundant `?v=` on its script tag.
- Plugin tabs show the manifest's `icon`: `/api/v3/plugins/installed` now includes it.
- `POST /api/v3/starlark/apps/<id>/toggle` goes through the same code as `/plugins/toggle`: `"false"` disables, a failed save no longer leaves the running app out of step with disk, and a loaded app with no manifest entry no longer answers 500.
- `/api/v3/` JSON responses are sent `Cache-Control: no-store`, so a reload right after an install, toggle or Wi-Fi connect shows the new state. Non-JSON files served through the API keep the 5 s cache.
- Startup plugin validation no longer gives up on a `null` plugin block, and plugins are discovered once at startup instead of twice.
- A plugin save drops repeated entries in lists whose schema says `uniqueItems`, instead of failing validation.
- `/api/v3/health` reports the real plugin count.
- A malformed `vegas_plugin_order` or `vegas_excluded_plugins` is refused with a 400 and nothing is saved. It used to wipe the saved list.
- The per-plugin health and metrics routes return the display service's latest state.
- Resetting a plugin's config takes a backup first and reports a failed save.
- System metrics that can't be read are `null` everywhere: `cpu_temp` off a Pi, and every metric without psutil, where `/system/status` now answers 200 instead of 503.
- `/plugins/store/refresh` no longer claims a commit-metadata refresh it doesn't do.
- The plugin-config list repair code is in one place, `src/web_interface/config_arrays.py`.
- Cache tab errors no longer show up in the Logs tab.
- A tab that fails to load shows "Try again" instead of a skeleton that never goes away.
- Plugin Store search and registry errors appear as a notification, and the Plugin Manager stays on screen.
- The image schedule button works on uploaded images, and the editor stays open while you edit.
- A failed plugin toggle moves the switch back.
- Each save shows one notification; a failed Durations save says it failed.
- Stats the server can't read show `--`.
- New `window.LEDEscape` (`html`, `attr`, `jsStringAttr`) replaces about 30 copied escapers. `window.escapeHtml` and `window.escapeAttribute` remain as aliases for plugin pages.
- The web service (`ledmatrix-web`) logs through `src.logging_config` like the
display service, so `journalctl -p err -u ledmatrix-web` works. Successful
GET/HEAD/OPTIONS requests (the UI's polling) are logged at DEBUG instead of
INFO; 4xx at WARNING, 5xx at ERROR. `LEDMATRIX_DEBUG=true` shows them again.
`web_interface/logging_config.py` is removed. The web cache
(`web_interface/cache.py`) now honours the TTL a value was stored with and is
thread-safe.
- `/api/v3` routes answer an exception they don't handle themselves from one
blueprint error handler, with the same `{status, message, details}` body the
53 removed per-route catch-alls returned. `ErrorCategory` and the
`error_category` key are removed from `src.web_interface.errors` (nothing read
them); `exception_error_response()` replaces the `from_exception` +
`error_response` pairs. A failing plugin action script's error now names the
real failure instead of `UnboundLocalError`.
- Installing a Starlark app works on a fresh install (#604). `starlark-apps/`
is created by whichever service reaches it first, and on a fresh install
that was usually the root display service, so the web interface could not
write to it and every install path answered "Failed to install from
repository". The display service now hands the directory and its contents
to the checkout's owner on every start (a no-op when not root or when the
checkout belongs to root), which also repairs devices already affected; a
permission error from the install routes names the directory and the fix.
- Clicks on plugin cards reach `handlePluginAction` (#605). Every click took
a copied fallback that asked twice before uninstalling and sent Starlark app
uninstalls to `POST /plugins/uninstall` instead of
`DELETE /starlark/apps/<id>`. A failed plugin toggle no longer always says
"A plugin operation is already in progress".
- The web interface starts with an absolute `plugin_system.plugins_directory`
(#616); it crashed at import with `NameError: project_root`.
- Stopping a Pixlet editor that ignores SIGTERM restarts the display instead
of answering 500 and leaving the panel dark (#625).
- Removed dead routes and files (#609): `POST /plugins/authenticate/spotify`
and `/ytm` (the music plugin runs its auth scripts through `web_ui_actions`),
`POST /plugins/of-the-day/json/upload` and `/json/delete` (they used the
wrong plugin id), `js/plugins/store_manager.js`, `js/config/diff_viewer.js`
and `js/htmx-sse.js`, and `web_interface/run.sh`. `htmx-config.js` no longer
replaces `console.error` / `console.warn`, which hid some real errors.
### Plugin error reporting
- `/api/v3/errors/summary` and `/api/v3/errors/plugin/<id>` report the errors
the display service recorded. They used to read the web process's own error
aggregator, which never records anything, so they always answered "no
errors". The display service now publishes a bounded snapshot to the shared
cache (`plugin_error_snapshot`, at most every 10 seconds and only on change;
`src/error_aggregator.py`, started from `DisplayController.__init__`).
Responses keep their shape and add `snapshot_available`, `generated_at` and
`clear_pending`; exception text has credentials redacted.
- `POST /api/v3/errors/clear` records a request (`plugin_error_clear_request`)
the display service applies within about 5 seconds; reads hide the cleared
errors at once. It accepts `"all": true`, and `cleared_count` can be `null`
when the count is only known to the display service.
- The Logs tab has a **Plugin errors** panel: per-plugin counts, repeating
errors and a Clear button.
- Credential redaction in exception text (`src/redaction.py`) takes time
proportional to the text, not its square. Two patterns were quadratic: URL
`user:password@`, on a long unbroken run of letters or digits (a hex digest,
an ID), and `Authorization:` followed by a long run of whitespace. Either
used to stall every thread of the display service for up to seconds each
time the snapshot was published: about 0.5s for 20k characters of hex, 8s
for 20k spaces. What gets redacted is unchanged.
### Wi-Fi
- WiFi status messages reach the panel (#605). The display controller looked
for `wifi_status.json` one directory above the repo; both sides now use
`wifi_manager.get_wifi_status_path()`, the file is written atomically, and
the plugin that resumes afterwards redraws the whole panel.
- The captive-portal checks (`/generate_204` and friends) also detect an access point brought up through NetworkManager, the fallback `enable_ap_mode` uses without hostapd; only hostapd was checked, so phones on that AP were told the internet worked.
- The WiFi monitor daemon re-reads `wifi_config.json` when it changes, so the "auto-enable AP mode" toggle takes effect without restarting the daemon.
- Disconnecting from WiFi in the web UI no longer runs an AP-mode check that could never enable the AP; it only added seconds of waiting. The daemon still enables the AP after its grace period.
- The WiFi status message file follows each WiFi manager's own config directory, and the config path falls back to this checkout rather than `/home/ledpi/LEDMatrix`.
- A wrong Wi-Fi password is reported as one again ("Incorrect password for ..."); the fallback that restores the old network or brings up the setup AP was replacing the signal.
- Wi-Fi disconnect takes the saved connection profile down.
- `wifi_config.json` is written atomically, and a save that fails now gets a 500.
### Fonts
- Fonts tab: the preview endpoint renders BDF fonts with the panel's own rasterizer instead of refusing them. (The Fonts page still skips the request for `.bdf`; enabling it there is a separate template change.)
- A plugin font declared as a `.zip` URL is served as the font extracted from it after a restart, instead of registering the archive itself. Font downloads time out after 30s and land in the cache only once complete, so an interrupted download is retried rather than served forever.
- BDF fonts: `FontManager.get_font()` and `element_style.load_font()` no longer hand one `freetype.Face` to every thread. BDF faces come from `load_bdf_face`, which already caches them per thread; TrueType fonts are cached as before. `element_style`'s font cache is locked (a concurrent eviction could raise `KeyError`).
- `FontManager.clear_cache()` and unregistering a plugin's fonts bump `cache_generation`, so cached layouts are rebuilt.
- `plugin://` fonts load from the plugin's own install directory. `FontManager.register_plugin_fonts()` takes an optional `plugin_dir`.
- Bundled font paths no longer depend on the directory the process was started from.
- `FontManager.get_font()` returns a BDF font at its native size when asked for
a size the file doesn't contain (5x7.bdf at 8 or 10px, say). It used to
return PIL's default font, a different typeface, so a plugin that relied on
that will now render the font it asked for.
- The web UI's Fonts tab has a **Used by** column: the loaded plugins that
registered each font with `FontManager.register_manager_font()`, published
by the display service to the shared cache (`src/font_usage.py`) and merged
into `GET /api/v3/fonts/catalog` as `used_by`. Deleting a font a plugin
uses now names those plugins in the confirmation (it is not blocked).
`FontManager.forget_manager_fonts()` is new; unloading a plugin calls it.
### Security
### Security (request paths and inline handlers, siblings of #561)
- `POST /api/v3/plugins/assets/upload`, `GET .../assets/list` and
`POST .../assets/delete` validate `plugin_id` with `src/common/path_safety`
@@ -634,24 +328,6 @@ core, the monorepo or the registry's third-party plugins calls them:
and add its own script. The store's View button opens only `http(s)` links.
- The uploaded-images list escapes each file's original name, path and ids; a
name like `<img src=x onerror=...>.png` was inserted as markup.
- Installing from a URL (and a registry install whose manifest renames the plugin) refuses a plugin id that isn't a single safe name, so `../x` can no longer delete and replace a directory outside the plugins directory.
- Plugin uninstall and config reset refuse core config sections (`display`, `schedule`, ...) and ids with path parts. Uninstall still cleans the config of a plugin whose directory is already gone.
- A config field marked `x-secret` whose value is an object or array is saved to `config_secrets.json`, not to `config.json` in plain text.
- Restoring a backup onto a device without `config_secrets.json`, `wifi_config.json` or `ytm_auth.json` creates them with mode 640 instead of world-readable 644.
- Backup export skips a plugin `manifest.json` that isn't a JSON object instead of failing, and two exports in the same second no longer share a temp file or overwrite each other (the second gets a `-2` suffix).
- Every font that ships in `assets/fonts/` is protected from deletion; `MatrixChunky8X`, `MatrixLight6X`, `MatrixLight8X` and `ic8x8u` could be deleted from the Fonts tab.
- The raw config and secrets editors, and endpoints using `validate_request_json`, answer 400 for a JSON body that isn't an object.
- `fix_web_permissions.sh` makes `safe_plugin_rm.sh` and `safe_pip_install.sh` root-owned again after resetting ownership. A web-user-owned copy of either is a root shell, since sudo lets the web user run them as root. It also restores `config_secrets.json` to mode 640.
- Wi-Fi passwords are no longer stored in `config/wifi_config.json` (#608).
`WiFiManager` appended every joined network's SSID and password, in plain
text, to `saved_networks`, and nothing read them back (NetworkManager keeps
its own credentials). Loading the config now drops a `saved_networks` key and
rewrites the file, so passwords already on disk are removed.
- The installers no longer grant the web user passwordless root on
`display_controller.py`, `start_display.sh` and `stop_display.sh` (#606).
Those files are owned by the user, so the web user could rewrite them and
run them as root; nothing ran them through sudo. Existing devices keep the
old rules until the installer or `configure_web_sudo.sh` is run again.
### Display hardware settings the library refuses
@@ -694,38 +370,6 @@ core, the monorepo or the registry's third-party plugins calls them:
(`legacy_bool_as_object` in `src/plugin_system/schema_manager.py`). Nothing
is written at load; the next save of that plugin's settings stores the object.
Other type mismatches still warn.
- A plugin that is reloaded (switched off and on again from the web UI) imports its own modules again, not another plugin's. Plugins import their own files by bare name (`from sports import ...`), which resolves to the first plugin directory on `sys.path` that has the file; the loader only added a directory that was missing, so a reloaded plugin's directory stayed behind any loaded since. On a Pi, re-enabling UFC with hockey running failed with "cannot import name '_status_is_final' from 'sports'". A loading plugin's directory is now always moved to the front.
- `src/plugin_system/store_manager.py` (2,977 lines) is split into mixins: `store_registry.py` (registry, GitHub metadata, search, manifest validation), `store_install.py` (install paths and dependencies) and `store_update.py` (updates, rollback, local git state). `PluginStoreManager` is still imported from `store_manager.py` and has exactly the same methods and attributes; every method body is byte-identical.
- Unloading a plugin waits (up to 5s) for an in-flight `update()` before running `cleanup()`/`on_disable()`, and an update that finishes after the unload no longer puts the plugin back to ENABLED.
- A plugin whose load fails after its module was imported (constructor, `validate_config()` or `on_enable()` raising) no longer leaves that module cached: fixing the plugin and reloading it runs the new code without a restart. Its font registrations are dropped too.
- `POST /api/v3/plugins/limits/<id>` answers 400 for a limit that isn't a non-negative number (a string limit used to make every later update of that plugin raise). A bad cached limits record is ignored with a warning instead of raising.
- The config schema is found for a plugin installed as `ledmatrix-<id>` or in a directory named differently from its manifest id, resolved the way the loader resolves it (plugins/ is still searched before plugin-repos/). A plugin with no schema is logged once at DEBUG instead of a warning on every lookup.
- Installing from a URL over an existing install sets the old copy aside and restores it if the move fails, under the same per-plugin lock as a registry install.
- The operation queue refuses a second operation for a plugin whose first is still waiting (a double-clicked Install ran twice), and no longer keeps every finished operation in memory.
- `get_vegas_render_width()` reads `display_manager.width` first, as plugins are told to.
- Store and state files are read as UTF-8 regardless of the system locale.
- Docs: `update_interval` in `config.json` sets the scheduler's cadence only for a plugin whose manifest has none (TROUBLESHOOTING, PLUGIN_CONFIGURATION_GUIDE). The health/metrics reset and limits routes note that they only change the web process's view.
- A plugin whose `on_enable()` raises is no longer left registered: the next load retries it instead of reporting "already loaded" for a plugin that never ran.
- One plugin's `get_info()` raising no longer breaks the installed-plugins list; it is logged and shown with empty runtime info.
- `plugin_state.json` and the operation history are written atomically (temp file + rename) under their lock, so concurrent saves or a failed save can't leave a truncated file.
- Plugin dependency installs run one `pip` at a time during parallel startup loading.
- A failed store download no longer leaves its extraction directory in the temp dir.
- Test doubles: `draw_image()` on `MockDisplayManager`, `VisualTestDisplayManager` and `BoundsCheckingDisplayManager` now emits a `DeprecationWarning` — the real `DisplayManager` has no such method; use `display_manager.image.paste(img, (x, y))`. `MockDisplayManager.draw_text` accepts the real signature's `small_font`/`centered` and default `x`/`y`, and `VisualTestDisplayManager` logs draw errors at WARNING.
- Removed the unused `PluginOperationQueue.get_active_operations()`.
- Updating a plugin that was installed from a ZIP no longer tries to reinstall it from the LEDMatrix repository's own URL.
- Repository URLs with `.git` in the middle are no longer mangled. The URL helpers now live in `src/plugin_system/repo_urls.py`.
- Installing from a URL works when the repository's only branch isn't `main` or `master`.
- A missing required config field is reported once, by name.
- A plugin that went over `max_memory_mb` once is no longer refused on every call after that.
- `reload_plugin` reads the manifest from the plugin's discovered directory.
- Removed: `last_display` from plugin state info and `get_last_display()` (nothing recorded them); `PluginOperationQueue`'s `history_file` and `lazy_load` arguments; and `data/plugin_operations.json`, which nothing read.
- One plugin-directory resolver, `src/plugin_system/plugin_dirs.py`, behind
discovery, `PluginManager.get_plugin_directory`, `PluginLoader`, the store and
state reconciliation. A manifest's `id` wins over a directory merely named for
the id; hidden and `.standalone-backup-` directories are never treated as
plugins (auto-update could previously try to update a backup); ids like
`a/b` or `..` resolve to nothing everywhere. Installs where each directory is
named for its manifest id, the installer's layout, behave as before.
### Core
@@ -744,39 +388,6 @@ core, the monorepo or the registry's third-party plugins calls them:
handling, so the restore stopped at `config.json` with nothing restored. The
ownership step is now skipped where `os.chown` is missing. No behaviour
change on the Pi.
- `APIHelper`'s rate limit and the display-sync heartbeat/leader timeouts measure elapsed time with `time.monotonic()`. A wall-clock step (NTP correcting a Pi with no RTC) could stall API requests for as long as the step or fake a sync timeout. `get_request_stats()['last_request_time']` is still wall-clock time.
- `sudo_remove_directory()` tries each bash path the sudoers rule might name, as `install_requirements_file()` already did.
- An element's saved layout `scale` equal to its schema default is no longer treated as a user choice when the default is declared under an alias (`score` for `score_text`).
- `CacheError`/`ConfigError`/`PluginError`/`DisplayError` no longer write their key into the caller's `context` dict; the JSON log formatter stringifies values it can't encode instead of dropping the record.
- Removed `ErrorAggregator`'s unused JSON export (`export_path`, `export_to_file()`); nothing called it. Docstring fixes in `validate_file_upload`, `StartupValidator.raise_on_errors`, `DisplaySyncManager.set_on_new_cycle`, `dynamic_team_resolver` and `config_arrays`.
- `/api/v3/errors` shows each exception's real stack trace instead of `NoneType: None`.
- Backups record `src.__version__`.
- Removed: `BackgroundDataService`'s `queue_size` stat and `clear_completed_requests()`.
- `src.device_location` — a blank `Location` field on a Starlark (Tidbyt) app
now renders at the device's City / State / Country (geocoded once via
Open-Meteo and cached) instead of the app author's hard-coded default,
usually San Francisco. A location saved on the app still wins. With no
device city set, or when the lookup fails or finds no match, the app keeps
its own default (a failed lookup is retried after 30 minutes). Clearing an
app's location in the web UI now actually clears it; the save used to drop
the blank field, so the old value stayed.
- Fixed a memory leak in the display service (#605): `ErrorAggregator`
appended every plugin in the time window to a pattern's `affected_plugins`
on each repeat (3,000 errors from three plugins reached 2.5 million
entries).
- An expired cache record is refused without being parsed (#633).
`CacheManager.set` writes `timestamp` and `ttl` ahead of `data`, and
`DiskCache.get` reads the first 256 bytes to decide staleness, with the same
rules as before. A 53 MB MLB season file used to be parsed in full (about
1.8 s holding the GIL on a Pi 4, freezing the display) only to be thrown
away. Files in the old layout are parsed as before and convert when
rewritten.
- Cache internals (#613): `CacheManager` delegates memory-tier cleanup and
stats to `MemoryCache`; `list_cache_files` no longer holds the memory lock
during directory I/O; `BackgroundDataService.get_sport_cache_key()` formats
the key instead of building a whole `CacheManager` (and probing the cache
directory) on every call; the unused request queue is gone, and `priority=`
is accepted and documented as ignored.
### Cache permissions
@@ -831,8 +442,6 @@ core, the monorepo or the registry's third-party plugins calls them:
timeouts with a second bash path, and all reinstalls share a 10-minute
budget, so a rollback finishes inside the unit's 30-minute limit instead of
being killed mid-way.
- A hand-edited non-object `auto_update` value reads as off instead of raising at startup, and a failed result write no longer leaves a temp file behind.
- Overview "Check Updates" asks for the same confirmation as "Update Code" and shows the server's message. Both, and the Tools tab's git pull, show the restart-pending banner when the update needs a restart.
### Installers
@@ -846,22 +455,6 @@ core, the monorepo or the registry's third-party plugins calls them:
`configure_web_sudo.sh` does the same before offering the rules for
confirmation. `first_time_install.sh` also built that file at a fixed `/tmp`
path as root; `mktemp` now picks the name.
- `check_system_compatibility.sh` treats Python 3.13 (what Trixie ships) as supported and anything below 3.10 as an error.
- `configure_web_sudo.sh` run as the web user keeps the reboot/poweroff rules.
- `check_system_compatibility.sh` no longer reports installed packages as missing.
- `configure_wifi_permissions.sh` checks its rules with `visudo -c` before installing them, and grants the NetworkManager captive-portal `cp` and `rm` commands `wifi_manager` runs.
- `configure_web_sudo.sh` uses a random temp file and installs its rules with mode 440.
- The installer prints its completion summary before the `-y` reboot, and describes the setup access point as an open network (it was shown with a password it doesn't have).
- `fix_cache_permissions.sh` applies `setup_cache.sh`'s `ledmatrix`-group model instead of setting 777.
- `check_system_compatibility.sh` reports anything but Debian 13 (Trixie) as unsupported, and reaches its summary.
- `one-shot-install.sh`'s `retry()` retries (#606). It read `$?` after `!`,
which is always 0, so a failed command ran once and was reported as a
success. It now tries three times and returns the command's status; both
apt steps still warn and continue after their retries, and a clone that
keeps failing stops the install sooner, with its own message.
- One generator for the web sudoers rules, `scripts/install/lib_sudoers.sh`,
used by `first_time_install.sh` and `configure_web_sudo.sh` (#622); the two
copies had drifted.
### Small fixes (update-all, plugin system settings, scripts)
@@ -899,9 +492,7 @@ core, the monorepo or the registry's third-party plugins calls them:
(only an explicit `web_display_autostart: false` keeps the web interface
down), so a missing key no longer shows as disabled. The shell scripts also
check `web_interface/blueprints/api_v3/`, which became a package, instead of
reporting `api_v3.py` as missing. (`scripts/verify_web_ui.sh`,
`scripts/diagnose_web_ui.sh` and `scripts/debug/debug_web_manual.py` were
later deleted as unreferenced; see Docs and developer tools.)
reporting `api_v3.py` as missing.
### Docs and developer tools
@@ -930,67 +521,6 @@ core, the monorepo or the registry's third-party plugins calls them:
`app.py` line numbers, `api_v3.py` paths, StreamManager method names,
nonexistent version-bump scripts and `ledmatrix` service user references
removed.
- A mypy ratchet in CI. `mypy-clean.txt` lists the 71 modules under `src/` that type-check clean, and the new "Type check (mypy ratchet)" job runs `python scripts/check_types.py` (mypy 1.20.2 on exactly those files) so they stay clean; add a module when you make it clean (see CONTRIBUTING.md). The manual pre-commit `mypy` hook runs the same script. 35 modules were made clean for it with annotation-only fixes, no behaviour change. Their public signatures only widened (`declared_min_version()` now says it returns the manifest's value as-is, `Any`); `DynamicTeamResolver._rankings_cache` is annotated as the abbreviation-to-rank dict it holds. `mypy.ini` treats numpy and orjson as `Any`, so it parses with `python_version = 3.10` against numpy 2.3+ stubs and gives the same result whether orjson is installed or not.
- CI runs the web UI's DOM test suites (jsdom against the real server-rendered pages and API) in a new **Web UI JS tests** job, with the web interface started in emulator mode; `REQUIRE_DOM=1` makes a suite that can't run fail instead of being skipped. Two suites that had gone stale were fixed: the Tools suite now installs `LEDEscape` the way `base.html` does and supplies sample Starlark apps when the server has none, and the Store suite no longer assumes the registry has 48 plugins or fewer.
- CI installs `web_interface/requirements.txt` too, so flask-limiter, flask-compress and the web floors are tested. `test_api_helper_does_not_hand_set_brotli` now checks what it meant: core doesn't add `br` itself, and `requests` may advertise it when a brotli decoder is installed.
- All Discord links point to the LEDMatrix server's invite.
- `pytz` may be any release before 2027, so current timezone data installs; `requirements-test.txt` caps `psutil` below 7 like the runtime requirements and allows `pytest-cov` up to 7.x (checked against pytest 9 with the CI coverage run).
- The Claude GitHub Actions workflows pin `anthropics/claude-code-action` to a commit SHA like the other actions.
- `mypy.ini` parses again. A multi-line `exclude` and trailing comments on values made mypy refuse the whole file, so none of its settings applied and the pre-commit hook failed with "Missing target". The mypy hook is now manual (`pre-commit run mypy --hook-stage manual`) while the ~500 existing type errors in `src/` are paid down.
- `.gitignore` ignores everything in `config/` except the templates; `ytm_auth.json`, `saved_repositories.json`, `wifi_status.json` and `font_overrides.json` weren't ignored.
- `.sh` and `.service` files are always checked out with LF line endings.
- The Claude code-review check is skipped on pull requests from forks, which get no secrets and always failed it.
- Doc fixes: emulator guide (Python 3.10+, `emulator_config.json` isn't in the repo), README's nonexistent "API Metrics" feature, a stale route count, and missing index entries for the scroll-performance and offscreen-rendering docs and the frame-soak and render-bench scripts.
- `src/common/README.md` lists `frame_timing`, `json_body` and `render_gate`.
- New `scripts/README.md` lists every script.
- New `docs/ARCHITECTURE.md` (processes, shared state, display loop, plugin system, web UI) and `docs/PERMISSIONS.md` (owners, modes, both sudoers files, repair scripts).
- Deprecated plugin APIs are marked in the plugin docs.
- `src/common/README.md` covers every module.
- Stale setup, service and troubleshooting claims are corrected.
- Deleted 13 scripts nothing referenced (#607): `utils/cleanup_venv.sh`,
`utils/clear_python_cache.sh`, `install/migrate_config.sh`,
`install/debug_install.sh`, `debug/debug_web_manual.py`,
`diagnose_web_ui.sh`, `verify_web_ui.sh`, `fix_internet_connectivity.sh`,
`diagnose_plugin_permissions.sh`, `dev/validate_python.py`,
`download_nba_logos.py` (with `README_NBA_LOGOS.md`) and
`setup_plugin_repos.py`, all under `scripts/`; also `docs/archive/` and
`PLUGIN_IMPLEMENTATION_SUMMARY.md`. `config.template.json` no longer carries
`plugin_system.auto_discover`, `auto_load_enabled` or `development_mode`,
which nothing reads (existing configs keep them). About 20 docs had stale
claims corrected against the code.
- `test/test_js_unit_suites.py` runs every `test/js/unit/*.js` suite under
pytest; CI used to run one of the eight (#605).
- New test `test/test_common_is_hardware_free.py`: `src/common` must import
without `rgbmatrix`, and never import `src.base_classes`,
`src.display_manager` or `src.plugin_system` at module level, so plugins can
use it on machines with no panel library.
### Removed
- **The skin system.** Skins never rendered with the current scoreboard
plugins, so they are gone rather than "not supported yet": `src/skin_system/`,
`skins/`, `scripts/validate_skin.py`, `GET /api/v3/skins`, the store's
`"type": "skin"` handling and `docs/SKIN_SYSTEM.md` / `docs/CREATING_SKINS.md`.
A `skin` or `skin_options` key left in a plugin's saved config still loads
and saves without a validation error; it is ignored, and the next save of
that plugin's settings removes it (unless the plugin's own schema declares
the key).
- **`src/base_classes/`** (`SportsCore`, the sport and mode classes,
`CelebrationMixin`, the rotation strategies, `data_sources`,
`api_extractors`). No known plugin imports it. A plugin that does must use
`src.common` or its own copy of the code.
- **Unused `src.common` modules and plugin-system helpers** (#608):
`src/common/config_helper.py`, `display_helper.py`, `game_helper.py`,
`utils.py` and `error_handler.py` (its re-exports leave `src.common`'s
`__all__`), `src/plugin_system/health_monitor.py` (`PluginHealthMonitor`,
whose loop did nothing; `PluginHealthTracker` is unchanged), and
`src.plugin_system.get_store_manager` / `__api_version__`. Nothing in core,
the scripts or the plugin monorepo imported them. `APIHelper`, `TextHelper`,
`ScrollHelper`, `LogoHelper` and the adaptive-layout exports of `src.common`
are unchanged. The same change removed unused methods from `ConfigService`,
`PluginStateManager`, `PluginManager`, `PluginExecutor`, `PluginLoader`,
`PluginStoreManager`, `VegasModeConfig` and `DisplayController`; none had
callers in core, the scripts or the monorepo.
## 3.4.0
+18 -11
View File
@@ -13,17 +13,14 @@
loader does NOT fall back to it — `PluginManager.discover_plugins()`
(`src/plugin_system/plugin_manager.py`) scans only the configured
directory. Fallbacks exist in two narrower places: store operations
(`PluginStoreManager._find_plugin_path()` in `store_manager.py`, which
searches `store_search_dirs()` from `plugin_dirs.py`) and schema lookup
(`SchemaManager.get_schema_path()` in `schema_manager.py`, which probes
`plugins/` *before* `plugin-repos/`).
- `src/plugin_system/plugin_dirs.py` — the one resolver for "which directory
holds plugin X" (manifest `id` first, then `<id>` / `ledmatrix-<id>`)
(`StoreManager._find_plugin_path()` in `store_manager.py`) and schema
lookup (`SchemaManager.get_schema_path()` in `schema_manager.py`,
which probes `plugins/` *before* `plugin-repos/`).
## Plugin System
- Plugins inherit from `BasePlugin` in `src/plugin_system/base_plugin.py`
- Required abstract methods: `update()`, `display(force_clear=False)`
- Each plugin needs: `manifest.json`, `config_schema.json`, and the entry point (`manager.py` by default); `requirements.txt` if it has dependencies. Required manifest fields: `docs/PLUGIN_API_REFERENCE.md#manifest-required-fields`
- Each plugin needs: `manifest.json`, `config_schema.json`, `manager.py`, `requirements.txt`
- Plugin instantiation args: `plugin_id, config, display_manager, cache_manager, plugin_manager`
- Config schemas use JSON Schema Draft-7
- Display dimensions: always read dynamically from `self.display_manager.width/height` — not `display_manager.matrix.width/height`, because `matrix` is `None` when hardware init fails (the properties fall back to the canvas size)
@@ -37,20 +34,30 @@
- Browser preview without the display loop: `python3 scripts/dev_server.py` → http://localhost:5001
- Full display in emulator mode: `python3 run.py -e` (or `EMULATOR=true python3 run.py`)
- Validate one plugin headlessly: `python3 scripts/check_plugin.py --plugin <id>`
- Soak a rig for frame timing (on the Pi, service running): `python3 scripts/frame_soak.py --preview` — late-frame rate across every scroller; see `docs/SCROLL_PERFORMANCE.md`
## Plugin Store Architecture
- Official plugins live in the `ledmatrix-plugins` monorepo (not individual repos)
- Plugin repo naming convention: `ledmatrix-<plugin-id>` (e.g., `ledmatrix-football-scoreboard`)
- `plugins.json` registry at `https://raw.githubusercontent.com/ChuckBuilds/ledmatrix-plugins/main/plugins.json`
- Store manager (`PluginStoreManager` in `src/plugin_system/store_manager.py`) handles install/update/uninstall
- Monorepo plugins are installed without a `.git` directory: GitHub Trees API + raw downloads, falling back to ZIP extraction
- Store manager (`src/plugin_system/store_manager.py`) handles install/update/uninstall
- Monorepo plugins are installed via ZIP extraction (no `.git` directory)
- Update detection for monorepo plugins uses version comparison (manifest version vs registry latest_version)
- Plugin configs stored in `config/config.json`, NOT in plugin directories — safe across reinstalls
- Third-party plugins can use their own repo URL with empty `plugin_path`
## Skin System (visual overlays for sports scoreboards) — NOT SUPPORTED YET
- Skins do not render with the current scoreboard plugins: the only hook is `SportsCore._render_game()` in `src/base_classes/sports/core.py`, and no current scoreboard plugin (monorepo or third-party registry) builds on `src.base_classes`
- So core doesn't offer them: no Visual Skin dropdown (`get_plugin_schema` skips `inject_skin_selector`), the store hides/refuses `"type": "skin"` entries, `GET /api/v3/skins` reports `"supported": false`. Switch: `SKINS_RENDER_SUPPORTED` in `src/skin_system/__init__.py`
- Stored `skin` / `skin_options` config values must keep loading and saving (base schema allows them; form saves deep-merge over the stored section)
- Skins live in `skins/<skin-id>/` (skin.json + skin.py), NOT in plugin dirs — plugin reinstall deletes plugin dirs
- Core: `src/skin_system/` (ScoreboardSkin, SkinContext, runtime); keep it and its tests
- Skins render onto `ctx.canvas` only; fallback to built-in renderer on `False`/exception (3 strikes disables for session)
- View-model guaranteed keys are frozen (see `test/test_skin_system.py::TestViewModelContract`) — renaming keys in `_extract_game_details_common` or sport extractors breaks published skins
- Validate skins headlessly: `python scripts/validate_skin.py --skin <id>`; docs: `docs/SKIN_SYSTEM.md`, `docs/CREATING_SKINS.md`
- Skins are NOT monorepo plugins: no manifest bump / update_registry.py needed
## Common Pitfalls
- paho-mqtt 2.x requires a `CallbackAPIVersion` argument: `VERSION1` for code written against v1 callback signatures (the MQTT bridge uses `VERSION2`)
- paho-mqtt 2.x needs `callback_api_version=mqtt.CallbackAPIVersion.VERSION1` for v1 compat
- BasePlugin uses `get_logger()` from `src.logging_config`, not standard `logging.getLogger()`
- `DisplayManager` has no `draw_image()` — paste onto the PIL image directly:
`self.display_manager.image.paste(img, (x, y))` then `update_display()`
+1 -1
View File
@@ -63,7 +63,7 @@ ChuckBuilds, and any other forums hosted by or affiliated with the project.
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement on the
[LEDMatrix Discord](https://discord.gg/RdrC37rEag) (DM a moderator or
[LEDMatrix Discord](https://discord.gg/uW36dVAtcT) (DM a moderator or
ChuckBuilds directly) or by opening a private GitHub Security Advisory if
the issue involves account safety. All complaints will be reviewed and
investigated promptly and fairly.
+4 -12
View File
@@ -9,7 +9,7 @@ improvements, and code changes.
- **Bugs / feature requests**: open an issue using one of the templates
in [`.github/ISSUE_TEMPLATE/`](.github/ISSUE_TEMPLATE/).
- **Real-time discussion**: the
[LEDMatrix Discord](https://discord.gg/RdrC37rEag).
[LEDMatrix Discord](https://discord.gg/uW36dVAtcT).
- **Plugin development**:
[`docs/PLUGIN_DEVELOPMENT_GUIDE.md`](docs/PLUGIN_DEVELOPMENT_GUIDE.md)
and the [`ledmatrix-plugins`](https://github.com/ChuckBuilds/ledmatrix-plugins)
@@ -58,18 +58,10 @@ integration tests.
3. **Keep PRs focused.** One conceptual change per PR. If you find
adjacent bugs while working, fix them in a separate PR.
4. **Follow the existing code style.** The pre-commit hooks run
`flake8` (E9, F63, F7, F82 plus bugbear `B` checks), `bandit`,
and `gitleaks` — install the CLI with
`flake8` (E9, F63, F7, F82 plus bugbear `B` checks), `mypy` on
`src/`, `bandit`, and `gitleaks` — install the CLI with
`python -m pip install pre-commit`, then run
`pre-commit install` so they run on every commit. Type checking
is a ratchet while the existing mypy errors in `src/` are paid
down: `mypy-clean.txt` lists the modules that type-check clean, and
CI runs `python scripts/check_types.py` (also the manual hook
`pre-commit run mypy --hook-stage manual`) to keep every listed
module clean. When you make another module clean, add it to the
list (sorted); don't take one off to get CI green. Keep type fixes
annotation-only where you can -- widen a hint rather than delete a
defensive runtime check mypy calls unreachable. HTML/JS in
`pre-commit install` so they run on every commit; HTML/JS in
`web_interface/` follows the patterns already in `templates/v3/`
and `static/v3/`.
5. **Update documentation** alongside code changes. If you add a
+1 -1
View File
@@ -44,7 +44,7 @@ Four strengths define LEDMatrix, and future work must protect all of them:
- **Hard constraint: plugin UI compatibility.** Third-party plugins rely on JSON Schema (Draft-7) generated config forms, the widget registry (`static/v3/js/widgets/`), `x-secret` fields, and plugin web-UI actions. UI changes must keep these working.
- **Config storage.** Plugin configuration lives in `config/config.json` and secrets in `config/config_secrets.json`, never in plugin directories, so configs survive reinstalls.
- **Stack.** An existing Flask + HTMX + Alpine.js app with Jinja templates (`web_interface/templates/v3/`) and static JS/CSS (`web_interface/static/v3/`), with self-hosted vendor assets.
- **Terminology.** Plugin, Plugin Store, Starlark app, rotation, display duration, Vegas Scroll Mode, on-demand, AP mode.
- **Terminology.** Plugin, Plugin Store, Starlark app, rotation, display duration, Vegas Scroll Mode, skin, on-demand, AP mode.
- **Open decisions** (offered during init, not adopted as constraints):
- Whether the UI must work fully offline, with no CDN fallbacks at runtime.
- Whether a Node/CSS build step is acceptable for contributors.
+21 -21
View File
@@ -33,7 +33,7 @@ I'm trying to be open to constructive criticism and support, as long as it's a r
- Show support on Youtube: https://www.youtube.com/@ChuckBuilds
- Check out the write-up on my website: https://www.chuck-builds.com/led-matrix/
- Stay in touch on Instagram: https://www.instagram.com/ChuckBuilds/
- Want to chat? Reach out on the LEDMatrix Discord: [https://discord.gg/RdrC37rEag](https://discord.gg/RdrC37rEag)
- Want to chat? Reach out on the LEDMatrix Discord: [https://discord.com/invite/uW36dVAtcT](https://discord.gg/dfFwsasa6W)
- Feeling Generous? Consider sponsoring this project or sending a donation (these AI credits aren't cheap!)
-----------------------------------------------------------------------------------
@@ -328,7 +328,6 @@ This one-shot installer will automatically:
- Install required system packages (git, python3, build tools, etc.)
- Clone or update the LEDMatrix repository
- Run the complete first-time installation script
- Print the web interface address, then **reboot the Pi automatically** (your SSH session will disconnect; give it a few minutes to come back)
The installation process typically takes 10-30 minutes depending on your internet connection and Pi model. Pi 3B/3B+ and other 1GB boards land at the top of that range, because the C++ library is compiled serially to stay within available memory. All errors are reported explicitly with actionable fixes.
@@ -461,6 +460,15 @@ See the [Plugin Store documentation](https://github.com/ChuckBuilds/ledmatrix-pl
For plugin development, the `plugins/hello-world/` plugin in the [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) repository is a starter template.
### Visual Skins for Scoreboards
**Not supported yet.** Skins are meant to restyle a sports scoreboard's
live/recent/upcoming screens without forking the plugin, but the current
scoreboard plugins don't render them: a selected skin has no effect. The web
UI doesn't offer skin install or selection for that reason. The skin system
and its docs stay in place for when scoreboards adopt it; see
[docs/SKIN_SYSTEM.md](docs/SKIN_SYSTEM.md) for why.
**Built-in Managers Deprecated**: The built-in managers (hockey, football, stocks, etc.) are now deprecated and have been moved to the plugin system. **You must install replacement plugins from the Plugin Store** in the web interface instead. The plugin system provides the same functionality with better maintainability and extensibility.
</details>
@@ -690,10 +698,9 @@ Controls how long each installed plugin stays visible in seconds before switchin
### Display Format Settings
- **`use_short_date_format`** (boolean, default: true)
- Currently has no effect. The web UI still saves it, but no core code
reads it. Scoreboard plugins that offer a short date format read the
setting from their own plugin config instead. See
[CONFIG_REFERENCE.md](docs/CONFIG_REFERENCE.md#display--other-keys).
- Use short date format (e.g., "Jan 15") instead of long format (e.g., "January 15th")
- Set to `false` for longer, more readable dates
- Set to `true` to save space and show more information
### Dynamic Duration Settings (`display.dynamic_duration`)
@@ -781,21 +788,15 @@ Controls how long each installed plugin stays visible in seconds before switchin
<details>
<summary>Manual SSH Commands (for reference)</summary>
The web interface's quick actions (Start/Stop/Restart Display) call
`sudo systemctl start|stop|restart ledmatrix.service` — see
`execute_system_action()` in
[`web_interface/blueprints/api_v3/system.py`](web_interface/blueprints/api_v3/system.py).
The service runs [`run.py`](run.py) as root.
The quick actions essentially just execute the following commands on the Pi.
To run the display in the foreground instead (for debugging), stop the service
first, then from the project root (e.g. `/home/ledpi/LEDMatrix`):
From the project root directory (ex: /home/ledpi/LEDMatrix):
```bash
sudo systemctl stop ledmatrix.service
sudo python3 run.py # add -d for debug logging
sudo python3 display_controller.py
```
This only runs as long as your SSH session stays open.
This will start the display cycle but only stays active as long as your ssh session is active.
### Convenience Scripts
@@ -948,7 +949,7 @@ sudo systemctl enable ledmatrix-web.service
- **On-Demand Controls**: Start specific displays (weather, stocks, sports) on demand
- **Service Management**: Start/stop the main display service
- **System Controls**: Restart, update code, and manage the system
- **System Stats**: CPU, memory and temperature on the Overview tab
- **API Metrics**: Monitor API usage and system performance
- **Logs**: View system logs in real-time
### Troubleshooting Web Interface
@@ -965,10 +966,9 @@ sudo systemctl enable ledmatrix-web.service
3. Check if another service is using port 5000
**Service Fails to Start:**
1. Check Python dependencies are installed. The installer puts them in the
system Python with `pip install --break-system-packages` (there is no
virtual environment), so `python3 -c "import flask"` should succeed.
2. Check file permissions and ownership
1. Check Python dependencies are installed
2. Verify the virtual environment is set up correctly
3. Check file permissions and ownership
</details>
+1 -1
View File
@@ -16,7 +16,7 @@ Use one of these channels, in order of preference:
maintainer.
- Direct link: <https://github.com/ChuckBuilds/LEDMatrix/security/advisories/new>
2. **Discord DM**. Send a direct message to a moderator on the
[LEDMatrix Discord](https://discord.gg/RdrC37rEag). Don't post in
[LEDMatrix Discord](https://discord.gg/uW36dVAtcT). Don't post in
public channels.
Please include:
+7 -15
View File
@@ -1,20 +1,12 @@
#!/usr/bin/env python3
"""Legacy entry point: runs ``run.py``, which is the one to use.
``python3 run.py`` (``-e`` for the emulator, ``-d`` for debug logging) is how
the display service and the docs start LEDMatrix. This file used to import
``src.display_controller.main`` directly, which skipped what run.py sets up
first -- ``sys.dont_write_bytecode`` (root-owned ``__pycache__`` in plugin
directories blocks the web service from updating them), the ``-e``/``-d``
flags, and the logging configuration. It now runs run.py exactly as
``python3 run.py`` would, with the same arguments.
"""
import os
import runpy
import sys
# Add the project root directory to Python path
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
from src.display_controller import main
if __name__ == "__main__":
runpy.run_path(
os.path.join(os.path.dirname(os.path.abspath(__file__)), "run.py"),
run_name="__main__",
)
main()
+73 -54
View File
@@ -185,52 +185,62 @@ their config section to control how oversized content is handled (see
### Plugin Integration (Developer Guide)
All of these have defaults in
[`BasePlugin`](../src/plugin_system/base_plugin.py); override only what you
need.
**1. Implement Content Method:**
```python
def get_vegas_content(self):
# Return a PIL Image, a list of Images, or None.
# A single image is one block; a list becomes one item per image.
return [self._render_game(game) for game in self.games]
```
"""
Return PIL Image or list of Images for Vegas mode.
If it returns `None` (the default), Vegas falls back to the plugin's
`scroll_helper` image, then to capturing `display()` output
(`PluginAdapter.get_content()` in
[`src/vegas_mode/plugin_adapter.py`](../src/vegas_mode/plugin_adapter.py)).
Returns:
PIL.Image or list[PIL.Image]: Content to display
- Single image: fixed-width content
- List of images: multiple segments
- None: skip this cycle
"""
# Example: Return single wide image
img = Image.new('RGB', (256, 32))
# ... render your content ...
return img
# Example: Return multiple segments
return [image1, image2, image3]
```
**2. Specify Content Type:**
```python
def get_vegas_content_type(self):
# 'multi' | 'static' | 'none' -- default is 'static'
return 'multi'
```
"""
Specify how content should be handled.
`'none'` excludes the plugin from Vegas mode.
Returns:
str: 'multi' | 'static' | 'none'
"""
return 'multi' # Default for most plugins
```
**3. Optionally Specify Display Mode:**
These return `VegasDisplayMode` members, not strings:
```python
from src.plugin_system.base_plugin import VegasDisplayMode
def get_vegas_display_mode(self):
return VegasDisplayMode.SCROLL
"""
Preferred display mode for this plugin.
Returns:
str: 'scroll' | 'fixed' | 'static'
"""
return 'scroll'
def get_supported_vegas_modes(self):
return [VegasDisplayMode.SCROLL, VegasDisplayMode.STATIC]
```
"""
List of supported modes.
`VegasDisplayMode` has `SCROLL` (`"scroll"`), `FIXED_SEGMENT` (`"fixed"`) and
`STATIC` (`"static"`). The default `get_vegas_display_mode()` uses the
plugin's `vegas_mode` config value if set, otherwise maps the content type
(`multi` to `SCROLL`, anything else to `FIXED_SEGMENT`).
Returns:
list: ['scroll', 'fixed', 'static']
"""
return ['scroll', 'static']
```
### Content Rendering Guidelines
@@ -956,16 +966,11 @@ from src.cache_manager import CacheManager
service = get_background_service(CacheManager())
stats = service.get_statistics()
print(f"Active: {stats['active_requests']}")
print(f"Completed: {stats['completed_requests']}")
print(f"Failed: {stats['failed_requests']}")
print(f"Active tasks: {stats['active_tasks']}")
print(f"Completed: {stats['completed']}")
print(f"Failed: {stats['failed']}")
```
Other keys: `total_requests`, `cached_hits`, `cache_misses`,
`average_fetch_time`, `completed_requests_count` (results currently held in
memory) — see `BackgroundDataService.get_statistics()` in
[`src/background_data_service.py`](../src/background_data_service.py).
**Enable Debug Logging:**
```python
import logging
@@ -976,10 +981,6 @@ logging.getLogger('src.background_data_service').setLevel(logging.DEBUG)
## 5. Permission Management
Ownership, modes, sudo rules and the repair scripts are listed in
[PERMISSIONS.md](PERMISSIONS.md). This section covers the helpers code uses
to keep files shareable.
### Overview
LEDMatrix uses a dual-user architecture: the display service runs as root (hardware access), while the web interface runs as a non-privileged user. Centralized permission management ensures both can access necessary files.
@@ -1043,7 +1044,7 @@ ensure_file_permissions(config_path, get_config_file_mode(config_path))
| Config (secrets) | `rw-r-----` | `0o640` | Owner write, group read |
| Assets | `rw-rw-r--` | `0o664` | Owner/group write, all read |
| Plugins | `rw-rw-r--` | `0o664` | Owner/group write, all read |
| Cache files | `rw-rw----` | `0o660` | Owner/group write, no world access (`_CACHE_FILE_MODE` in `src/cache/disk_cache.py`) |
| Cache files | `rw-rw-r--` | `0o664` | Owner/group write, all read |
**Directory Permissions:**
@@ -1114,22 +1115,40 @@ These core utilities **already handle permissions** - you don't need to call per
### Manual Fixes
[PERMISSIONS.md](PERMISSIONS.md) lists who owns what on an installed system,
the expected modes, and which `scripts/fix_perms/` script to run as which
user. In short:
If you encounter permission issues:
- `fix_assets_permissions.sh`, `fix_cache_permissions.sh` and
`fix_plugin_permissions.sh` are run with `sudo`.
- `fix_web_permissions.sh` is run as the web interface user, without
`sudo` (it refuses to run as root and calls `sudo` itself where needed).
It resets project file ownership for that user, then makes the two
helper scripts the web user may run as root (`safe_plugin_rm.sh`,
`safe_pip_install.sh`) root-owned again and restores `config_secrets.json`
to its owner, the `ledmatrix` group and mode `640`. It does not write
sudoers rules; `scripts/install/configure_web_sudo.sh` does that.
```bash
# Targeted permission fixes (see scripts/fix_perms/README.md)
sudo ./scripts/fix_perms/fix_assets_permissions.sh # assets/ tree (logos, fonts)
sudo ./scripts/fix_perms/fix_cache_permissions.sh # all cache directories
sudo ./scripts/fix_perms/fix_plugin_permissions.sh # plugin directories
sudo ./scripts/fix_perms/fix_web_permissions.sh # web interface files
Do not `chmod` the whole `config/` directory: `config_secrets.json` must stay
`640`.
# Fix specific directory
sudo chown -R ledpi:ledpi /home/ledpi/LEDMatrix/config
sudo chmod -R 2775 /home/ledpi/LEDMatrix/config
sudo find /home/ledpi/LEDMatrix/config -type f -exec chmod 664 {} \;
# Verify permissions
ls -la config/
ls -la assets/
```
### Verification
```bash
# Check directory has setgid bit
ls -ld assets/
# Should show: drwxrwsr-x (note the 's')
# Check file has correct group
ls -l assets/logo.png
# Should show group 'ledpi'
# Check file permissions
stat -c "%a %n" config/config.json
# Should show: 644 config/config.json
```
---
+79 -17
View File
@@ -13,7 +13,7 @@ Advanced patterns, examples, and best practices for developing LEDMatrix plugins
- [Using Weather Icons](#using-weather-icons)
- [Implementing Scrolling with Deferred Updates](#implementing-scrolling-with-deferred-updates)
- [Cache Strategy Patterns](#cache-strategy-patterns)
- [Font Management](#font-management)
- [Font Management and Overrides](#font-management-and-overrides)
- [Error Handling Best Practices](#error-handling-best-practices)
- [Performance Optimization](#performance-optimization)
- [Testing Plugins with Mocks](#testing-plugins-with-mocks)
@@ -25,12 +25,69 @@ Advanced patterns, examples, and best practices for developing LEDMatrix plugins
## Using Weather Icons
The Display Manager's icon methods — `draw_weather_icon()`, `draw_sun()`,
`draw_cloud()`, `draw_rain()`, `draw_snow()` and `draw_text_with_icons()` —
are deprecated, removed in 3.7.0. Draw your own icons instead: render them
onto a PIL image and paste it onto `self.display_manager.image`, or ship
icon images with the plugin. The weather plugin's `WeatherIcons` class is an
example. See [Deprecated APIs](PLUGIN_API_REFERENCE.md#deprecated-apis).
The Display Manager provides built-in weather icon drawing methods for easy visual representation of weather conditions.
### Basic Weather Icon Usage
```python
def display(self, force_clear=False):
if force_clear:
self.display_manager.clear()
# Draw weather icon based on condition
condition = self.data.get('condition', 'clear')
self.display_manager.draw_weather_icon(condition, x=5, y=5, size=16)
# Draw temperature next to icon
temp = self.data.get('temp', 72)
self.display_manager.draw_text(
f"{temp}°F",
x=25, y=10,
color=(255, 255, 255)
)
self.display_manager.update_display()
```
### Supported Weather Conditions
The `draw_weather_icon()` method automatically maps condition strings to appropriate icons:
- `"clear"`, `"sunny"` → Sun icon
- `"clouds"`, `"cloudy"`, `"partly cloudy"` → Cloud icon
- `"rain"`, `"drizzle"`, `"shower"` → Rain icon
- `"snow"`, `"sleet"`, `"hail"` → Snow icon
- `"thunderstorm"`, `"storm"` → Storm icon
### Custom Weather Icons
For more control, use individual icon methods:
```python
# Draw specific icons
self.display_manager.draw_sun(x=10, y=10, size=16)
self.display_manager.draw_cloud(x=10, y=10, size=16, color=(150, 150, 150))
self.display_manager.draw_rain(x=10, y=10, size=16)
self.display_manager.draw_snow(x=10, y=10, size=16)
```
### Text with Weather Icons
Use `draw_text_with_icons()` to combine text and icons:
```python
icons = [
("sun", 5, 5), # Sun icon at (5, 5)
("cloud", 100, 5) # Cloud icon at (100, 5)
]
self.display_manager.draw_text_with_icons(
"Weather: Sunny, Cloudy",
icons=icons,
x=10, y=20,
color=(255, 255, 255)
)
```
---
@@ -194,8 +251,11 @@ def update(self):
sport_key = "nhl"
cache_key = f"{self.plugin_id}_{sport_key}_games"
# get_background_cached_data() is deprecated, removed in 3.7.0 — use get()
cached = self.cache_manager.get(cache_key, max_age=60)
# Uses sport-specific live_update_interval from config
cached = self.cache_manager.get_background_cached_data(
cache_key,
sport_key=sport_key
)
if cached:
self.games = cached
@@ -222,9 +282,9 @@ def on_config_change(self, new_config):
---
## Font Management
## Font Management and Overrides
The display manager's built-in fonts and text measurement. For fonts shipped with a plugin, see [FONT_MANAGER.md](FONT_MANAGER.md).
Use the Font Manager for advanced font handling and user customization.
### Using Different Fonts
@@ -596,12 +656,14 @@ def update(self):
```python
def update(self):
# get_enabled_plugins() is deprecated, removed in 3.7.0 — check the
# instance's `enabled` flag instead
weather_plugin = self.plugin_manager.get_plugin("weather")
if weather_plugin is not None and weather_plugin.enabled:
# Use weather data
pass
# Check if another plugin is enabled
enabled_plugins = self.plugin_manager.get_enabled_plugins()
if "weather" in enabled_plugins:
# Weather plugin is available
weather_plugin = self.plugin_manager.get_plugin("weather")
if weather_plugin:
# Use weather data
pass
```
### Sharing Data Between Plugins
-220
View File
@@ -1,220 +0,0 @@
# Architecture
A map of the codebase for a new contributor: which process does what, how
they talk to each other, and where to start reading for common changes.
## Processes
| systemd unit | Runs as | Runs | Installed by |
|---|---|---|---|
| `ledmatrix.service` | root | [`run.py`](../run.py) → `DisplayController` | [`install_service.sh`](../scripts/install/install_service.sh) |
| `ledmatrix-web.service` | the installing user | [`start_web_conditionally.py`](../scripts/utils/start_web_conditionally.py) → [`web_interface/start.py`](../web_interface/start.py) (Flask, port 5000) | `install_service.sh`, [`install_web_service.sh`](../scripts/install/install_web_service.sh) |
| `ledmatrix-update-verify.path` / `.service` | the web user | Health check after an automatic update | the same installers, or [`src/auto_update_setup.py`](../src/auto_update_setup.py) at runtime |
| `ledmatrix-wifi-monitor.service` | root | [`wifi_monitor_daemon.py`](../scripts/utils/wifi_monitor_daemon.py) | [`install_wifi_monitor.sh`](../scripts/install/install_wifi_monitor.sh) |
| `ledmatrix-mqtt-bridge.service` | root | [MQTT bridge](../integrations/mqtt_bridge/README.md) (optional) | [`install_mqtt_bridge.sh`](../scripts/install/install_mqtt_bridge.sh) |
| `ledmatrix-dns-fix.service` | root | DNS workaround (optional) | [`install_dns_fix.sh`](../scripts/install/install_dns_fix.sh) |
Unit templates are in [`systemd/`](../systemd/README.md). The display runs as
root because the LED matrix library needs direct GPIO access. The web
interface runs unprivileged and uses a fixed list of `sudo` rules for the
few privileged things it does; see [PERMISSIONS.md](PERMISSIONS.md).
`start_web_conditionally.py` exits without starting Flask when
`web_display_autostart` is explicitly false in `config.json`.
## How the two main processes share state
The display and the web interface are separate processes that never call
each other. They share three things:
1. **`config/config.json` and `config/config_secrets.json`.** The web
interface writes them through `ConfigManager`
([`src/config_manager.py`](../src/config_manager.py)); the display
notices through `ConfigService` (below).
2. **The disk cache**, `/var/cache/ledmatrix` (owned `root:ledmatrix`,
setgid, files `0660`), read and written through `CacheManager`
([`src/cache_manager.py`](../src/cache_manager.py),
[`src/cache/disk_cache.py`](../src/cache/disk_cache.py)). Readers in the
other process pass `memory_ttl=0` so they do not serve a stale in-memory
copy.
3. **A few files in `/tmp`.**
| State | Where | Written by | Read by |
|---|---|---|---|
| On-demand request | cache `display_on_demand_request` | web: `start_on_demand_display()` / `stop_on_demand_display()` in [`api_v3/display.py`](../web_interface/blueprints/api_v3/display.py) | display: `_poll_on_demand_requests()` |
| On-demand state | cache `display_on_demand_state` | display: `_publish_on_demand_state()` | web: `/api/v3/display/on-demand/status` |
| Current screen | cache `display_current_state` | display | web: `/api/v3/display/current-status` |
| Plugin errors | cache `plugin_error_snapshot` | display: `ErrorSnapshotPublisher` ([`src/error_aggregator.py`](../src/error_aggregator.py)) | web: `read_error_report()` for `/api/v3/errors/*` |
| Error clear | cache `plugin_error_clear_request` | web | display |
| Font usage | cache `font_usage_snapshot` | display: `FontUsagePublisher` ([`src/font_usage.py`](../src/font_usage.py)) | web: Fonts tab |
| Plugin health | cache `plugin_health:<id>` | display (web writes on reset) | web: `/api/v3/plugins/health` |
| Preview frame | `/tmp/led_matrix_preview.png` | display: `DisplayManager`, gated by [`snapshot_policy`](../src/common/snapshot_policy.py) | web: display SSE stream, `/api/v3/health` (file age) |
| Preview viewer marker | `/tmp/led_matrix_preview_viewer` | web, while a preview is open | display: writes full-rate snapshots only while it is fresh |
| Hardware init status | `/tmp/led_matrix_hw_status.json` | display | web: `/api/v3/hardware/status` |
The on-demand start route also restarts `ledmatrix.service` by default so the
request takes effect straight away.
## Display loop
[`src/display_controller.py`](../src/display_controller.py), class
`DisplayController`. `__init__` loads config, starts the cache and the
error-snapshot publisher, runs the startup validator, creates the
`DisplayManager` ([`src/display_manager.py`](../src/display_manager.py)),
`FontManager` and `PluginManager`, loads the enabled plugins in parallel,
runs an initial `update()` pass within a 20-second budget
(`_INITIAL_UPDATE_BUDGET_SECONDS`; a plugin that misses it is deferred to
the scheduler), and sets up Vegas mode.
`run()` is the main loop. Each pass, in order: apply a pending plugin
enable/disable, poll on-demand requests, run scheduled plugin updates, check
the on/off schedule and brightness, then show one screen. Priority is
on-demand, then WiFi status messages, then live priority, then Vegas mode,
then normal rotation.
- **Rotation.** `available_modes` is the ordered list of display modes;
`current_mode_index` advances after each screen.
`_apply_plugin_rotation_order()` applies `display.plugin_rotation_order`.
- **Durations.** `_get_display_duration()`: `display.display_durations[mode]`,
else the plugin's `get_display_duration()`, else 30 s. Plugins that
support dynamic duration run until `is_cycle_complete()`, capped by
`display.dynamic_duration.max_duration_seconds` (default 180 s).
- **On-demand.** A request from the web interface pins one plugin (or mode)
for a duration. `_activate_on_demand()` / `_clear_on_demand()`; the
session is saved under `display_on_demand_config` so it survives a
restart. It also keeps the display on during scheduled off hours.
- **Live priority.** `_check_live_priority()` looks for a plugin whose
`has_live_priority()` and `has_live_content()` are both true and switches
to it, rotating between several live games.
- **Schedule and dim schedule.** `_check_schedule()` reads `schedule`;
`_check_dim_schedule()` reads `dim_schedule` and
`display.hardware.brightness`. Both are re-evaluated once a minute.
- **Long screens.** While a screen is showing (a dwell, a scroll, a Vegas
iteration), `_service_pending_changes()` repeats the on-demand, schedule
and brightness checks every 0.25 s, so a change does not wait for the
screen to end.
- **Config hot reload.** `ConfigService`
([`src/config_service.py`](../src/config_service.py)) polls the config and
secrets files' mtimes every 2 s and notifies subscribers when the content
changes. The controller refreshes its cached settings; enabling or
disabling a plugin queues `_reconcile_enabled_plugins()`, which loads or
unloads it on the display thread; each plugin gets `on_config_change()`
for its own section. Set `LEDMATRIX_HOT_RELOAD=false` to turn this off.
Matrix hardware settings are only read at start-up.
- **Vegas mode.** [`src/vegas_mode/`](../src/vegas_mode/): the display loop
calls `VegasModeCoordinator.run_iteration()`
([`coordinator.py`](../src/vegas_mode/coordinator.py)) when
`display.vegas_scroll.enabled` is set. `PluginAdapter` gets each plugin's
content (`get_vegas_content()`, else its `scroll_helper` image, else a
capture of `display()`), `StreamManager` orders it and `RenderPipeline`
scrolls it. See [ADVANCED_FEATURES.md](ADVANCED_FEATURES.md).
- **Multi-display sync.** `DisplaySyncManager`
([`src/common/sync_manager.py`](../src/common/sync_manager.py)), enabled by
`sync.role`: a leader sends a follower its share of each frame over UDP
(port 5765).
## Plugin system
[`src/plugin_system/`](../src/plugin_system/):
| Area | Where |
|---|---|
| Base class plugins implement | [`base_plugin.py`](../src/plugin_system/base_plugin.py) (`BasePlugin`, `VegasDisplayMode`) |
| Finding a plugin's directory | [`plugin_dirs.py`](../src/plugin_system/plugin_dirs.py): manifest `id` first, then directory `<id>` or `ledmatrix-<id>` |
| Discovery, load, unload, scheduled updates | [`plugin_manager.py`](../src/plugin_system/plugin_manager.py) (`PluginManager`) |
| Import and instantiate | [`plugin_loader.py`](../src/plugin_system/plugin_loader.py) (`PluginLoader.load_plugin()`: dependencies, module, class) |
| Timeouts | [`plugin_executor.py`](../src/plugin_system/plugin_executor.py) (`PluginExecutor`, 30 s default; a timed-out thread is abandoned, not killed) |
| Circuit breaker | [`plugin_health.py`](../src/plugin_system/plugin_health.py) (`PluginHealthTracker`: 3 consecutive failures open the circuit for 300 s) |
| Resource metrics | [`resource_monitor.py`](../src/plugin_system/resource_monitor.py) |
| Config schemas and defaults | [`schema_manager.py`](../src/plugin_system/schema_manager.py) |
| Install, update, uninstall | [`store_manager.py`](../src/plugin_system/store_manager.py) (`PluginStoreManager`), with its methods split across [`store_registry.py`](../src/plugin_system/store_registry.py) (registry, GitHub), [`store_install.py`](../src/plugin_system/store_install.py) and [`store_update.py`](../src/plugin_system/store_update.py) |
| Core-version gate | [`compatibility.py`](../src/plugin_system/compatibility.py) |
Discovery scans only `plugin_system.plugins_directory` (default
`plugin-repos/`). Scheduled `update()` calls run on one background worker
thread; a per-plugin lock keeps `display()` from running during an update.
**Store flow.** `install_plugin()` renames any existing copy aside
(`<id>.standalone-backup-preinstall`), installs the new one, and puts the old
copy back if the install fails. Monorepo plugins come from the GitHub Trees
API, falling back to the repository ZIP; other plugins by `git clone` or
download. The manifest is checked (see
[required fields](PLUGIN_API_REFERENCE.md#manifest-required-fields)), the core
version gate runs, then dependencies are installed as root through
`scripts/fix_perms/safe_pip_install.sh`. `update_plugin()` pulls git
installs, undoing a pull whose new version is incompatible, and reinstalls
everything else through `_reinstall_with_rollback()`.
## Web interface
- **App.** [`web_interface/app.py`](../web_interface/app.py) builds the
Flask `app` at import time, creates the managers, and registers two
blueprints. `web_interface/start.py` runs it on port 5000.
- **Pages.** [`blueprints/pages_v3.py`](../web_interface/blueprints/pages_v3.py)
serves the shell `templates/v3/base.html` at `/` and each tab as a
partial at `/partials/<name>` (templates in
`web_interface/templates/v3/partials/`). Plugin configuration tabs are
rendered from the plugin's schema by `plugin_config.html`.
- **API.** [`blueprints/api_v3/`](../web_interface/blueprints/api_v3/) is one
blueprint at `/api/v3`, split by area: `backup.py`, `config.py`,
`display.py`, `fonts.py`, `misc.py` (health, logs, errors, cache, sync),
`starlark.py`, `system.py` (service actions, updates, git), `wifi.py`, and
the plugin routes: `plugins.py` (installed list, enable/disable, plugin
actions), `plugin_store.py` (install, update, uninstall, store),
`plugin_config.py` (config, schema, reset), `plugin_assets.py` (uploads,
plugin static files), `plugin_health.py` (health, metrics, limits),
`plugin_operations.py` (operation history, state reconciliation) and
`plugin_calendar.py`. `__init__.py` defines the blueprint and shared helpers and
imports the modules so their routes register. Endpoints are listed in
[REST_API_REFERENCE.md](REST_API_REFERENCE.md).
- **Front end.** HTMX loads each tab's partial on first open
(`hx-trigger="loadtab"`); Alpine.js holds page state. Scripts are in
`web_interface/static/v3/js/`; form widgets are bundled from
[`js/widgets/`](../web_interface/static/v3/js/widgets/README.md).
- **Server-sent events** (`app.py`): `/api/v3/stream/stats` (CPU, memory,
temperature, service state, every 10 s), `/api/v3/stream/display` (preview
frames when the PNG changes) and `/api/v3/stream/logs` (journal of both
services). One generator thread per stream is shared by all clients.
## Updates
- **Update Code** on the Overview tab and the automatic updater both call
`perform_core_update()` in
[`api_v3/system.py`](../web_interface/blueprints/api_v3/system.py):
`git pull --rebase`, reinstall changed requirement files, report whether a
restart is needed.
- **Automatic updates** (`auto_update.enabled`, off by default):
`AutoUpdater` in [`web_interface/auto_update.py`](../web_interface/auto_update.py)
runs in the web process, checks every 30 minutes, and updates at most
weekly between 02:00 and 05:00. Before pulling it copies
[`scripts/utils/auto_update_verify.py`](../scripts/utils/auto_update_verify.py)
to `data/auto_update_verifier.py`, then writes
`data/auto_update_verify.request`. That file triggers
`ledmatrix-update-verify.path`, which runs the verifier as a separate unit
(so restarting the web service does not kill it). The verifier restarts
both services, waits for the web API to answer and the display service to
stay up, and on failure resets to the previous commit and restarts again.
Plugin updates run only after a verified core update. State is in
`data/auto_update_state.json` and `data/auto_update_pending.json`.
- **Startup validator.** `StartupValidator`
([`src/startup_validator.py`](../src/startup_validator.py)) runs twice in
`DisplayController.__init__`: config and cache directory first, then
enabled plugins once the plugin manager exists. It also warns when an
installed systemd unit differs from its template in `systemd/`. Results
are logged; startup continues either way.
## Where to start reading
| Task | Start with |
|---|---|
| Change rotation, durations or priorities | `DisplayController.run()` and `_get_display_duration()` in [`display_controller.py`](../src/display_controller.py) |
| Add a config key | [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md), [`config/config.template.json`](../config/config.template.json), the tab's partial and `api_v3/config.py` |
| Change drawing or fonts | [`display_manager.py`](../src/display_manager.py), [`font_manager.py`](../src/font_manager.py), [`src/common/bdf_font.py`](../src/common/bdf_font.py) |
| Add a plugin-facing API | [`base_plugin.py`](../src/plugin_system/base_plugin.py) or [`src/common/`](../src/common/README.md); document it in [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md) |
| Plugin install/update bugs | `PluginStoreManager` in [`store_manager.py`](../src/plugin_system/store_manager.py) |
| A plugin that won't load | `PluginManager.load_plugin()` and `PluginLoader.load_plugin()`; `python3 scripts/check_plugin.py --plugin <id>` |
| Add an API endpoint | the matching module in [`api_v3/`](../web_interface/blueprints/api_v3/) |
| Add a web UI tab or control | `templates/v3/base.html`, the tab's partial, `pages_v3.py` |
| Vegas scroll | [`src/vegas_mode/coordinator.py`](../src/vegas_mode/coordinator.py) |
| Installer or permissions | [`first_time_install.sh`](../first_time_install.sh), [`scripts/install/`](../scripts/install/), [PERMISSIONS.md](PERMISSIONS.md) |
| Work without a Pi | [DEV_PREVIEW.md](DEV_PREVIEW.md), [EMULATOR_SETUP_GUIDE.md](EMULATOR_SETUP_GUIDE.md), [HOW_TO_RUN_TESTS.md](HOW_TO_RUN_TESTS.md) |
+5 -11
View File
@@ -172,14 +172,10 @@ ERROR - Plugin football-scoreboard configuration validation failed: 'api_key' is
### Enable Debug Logging
Run the display in the foreground with `-d`, or set `LEDMATRIX_DEBUG=true`
(the value must be `true`; `1` is ignored — see `setup_logging()` in
[`src/logging_config.py`](../src/logging_config.py)):
Set environment variable:
```bash
sudo systemctl stop ledmatrix.service
sudo python3 run.py -d
# or
sudo LEDMATRIX_DEBUG=true python3 run.py
export LEDMATRIX_DEBUG=1
python run.py
```
### Check Merged Configuration
@@ -325,10 +321,8 @@ cp config/backups/config.json.backup.20240115_120000_000000 config/config.json
## Getting Help
1. Check logs. Both services log to journald, not to a file:
`sudo journalctl -u ledmatrix.service -f` (display) and
`sudo journalctl -u ledmatrix-web.service -f` (web interface)
2. Enable debug: `LEDMATRIX_DEBUG=true` or `python3 run.py -d`
1. Check logs: `tail -f logs/ledmatrix.log`
2. Enable debug: `LEDMATRIX_DEBUG=1`
3. Check error dashboard: `/api/v3/errors/summary`
4. Validate JSON: https://jsonlint.com/
5. File an issue: https://github.com/ChuckBuilds/LEDMatrix/issues
+4 -9
View File
@@ -19,7 +19,7 @@ tooling against it.
| `auto_update.enabled` | bool, `false` | Weekly automatic updates: LEDMatrix code first (health-checked, rolled back on failure), then installed plugins. Toggle in the General tab or install with `first_time_install.sh --enable-auto-update` | `web_interface/auto_update.py`, `src/auto_update_setup.py` (`is_enabled()`) |
| `timezone` | string, `"America/New_York"` | IANA timezone for schedules and displays | `ConfigManager.get_timezone()` |
| `target_fps` | int, `100` | Legacy "Scroll Frame Rate". Core scrolling no longer reads it: scroll frames are presented at `display.hardware.limit_refresh_rate_hz` divided by each scroll's frame hold, and speed comes from each plugin's scroll settings. Still exposed to plugins via `BasePlugin.global_config` | `src/plugin_system/base_plugin.py` |
| `location` | object | `city` / `state` / `country`. Supplies the **default** for a plugin's own `location_city` / `location_state` / `location_country` setting, so weather, radar and friends follow this device without being configured twice. A value saved on the plugin itself still overrides it. Starlark (Tidbyt) apps get the same treatment: a `Location` field left blank on the app renders at this city (geocoded once via Open-Meteo, coordinates cached permanently) instead of the app author's default, which is usually San Francisco. If the city can't be looked up (no match, or the geocoder is unreachable; retried after 30 minutes), the app keeps its own default. | `SchemaManager.apply_device_location()`, then plugins via merged config; `src/device_location.py` for Starlark apps |
| `location` | object | `city` / `state` / `country`. Supplies the **default** for a plugin's own `location_city` / `location_state` / `location_country` setting, so weather, radar and friends follow this device without being configured twice. A value saved on the plugin itself still overrides it. | `SchemaManager.apply_device_location()`, then plugins via merged config |
## `schedule` — display on/off hours
@@ -104,8 +104,7 @@ logical image to multiple chained physical panels.
|---|---|---|---|
| `display_durations` | object, `{}` | Per-plugin display duration in seconds, keyed by plugin id (e.g. `"clock": 15`) | `DisplayController._get_display_duration()` (`src/display_controller.py`) |
| `plugin_rotation_order` | array, `[]` | Explicit rotation order of plugin ids; empty = all enabled plugins in discovery order | `DisplayController._apply_plugin_rotation_order()` (`src/display_controller.py`) |
| `use_short_date_format` | bool, `true` | Compact date rendering in sports scoreboards | Nothing since `src/base_classes` was removed; scoreboards read `display.use_short_date_format` from their own plugin config |
| `scan_order_compensation` | string, `"auto"` | `"auto"` shows one half of each panel a refresh behind while something scrolls at one frame per refresh, which removes the 1px step a 1:N-scan panel shows across its middle; `"off"` disables it. Applies only to layouts whose row order is known: plain or parallel chains, 0 or 180 degree orientation, `multiplexing` 0, `scan_mode` 0, and not in the emulator | `DisplayManager._setup_scan_order_compensation()` (`src/display_manager.py`, `src/scan_order.py`) |
| `use_short_date_format` | bool, `true` | Compact date rendering in sports scoreboards | `src/base_classes/sports/core.py` |
| `dynamic_duration.max_duration_seconds` | int, optional | Cap for plugins that request dynamic display time | `DisplayController._get_global_dynamic_cap()` (`src/display_controller.py`) |
## `display.vegas_scroll` — continuous scroll mode
@@ -128,11 +127,7 @@ Read by `src/vegas_mode/config.py` (`VegasScrollConfig.from_config`). See
| `min_content_separation` | int, `24` |
| `min_cut_gap` | int, `6` |
| `continuous_scroll` | bool, `true` |
| `offscreen_prefetch` | bool, `true` — render every plugin's ticker content on the background thread, each on its own canvas. `false` restores handing canvas-bound plugins to the render thread, one pause at a time. Temporary; see [OFFSCREEN_RENDERING.md](OFFSCREEN_RENDERING.md) |
| `prefetch_gate` | bool, `true` — let that background thread run Python only while the render thread is waiting for the panel, so the render thread never waits for the GIL when a refresh comes round. Only takes effect with the rebuilt rgbmatrix binding (`scripts/build_rgbmatrix_nogil.sh`). See [OFFSCREEN_RENDERING.md](OFFSCREEN_RENDERING.md) |
| `switch_interval_ms` | float, `0` — experimental: shorten Python's GIL switch interval to this many ms while Vegas runs. `0` leaves the default (5 ms) alone |
| `smooth_scroll` | bool, `true` — move a whole number of pixels per panel refresh, locked to vsync. `scroll_speed` is snapped to the nearest speed the panel can show that way (at 95Hz: 95, 47.5, 31.7 px/s…), measured against the panel's real refresh rate once scrolling starts |
| `sub_pixel_blend` | bool, `false` — the older smoothing: advance by elapsed time and blend neighbouring pixel columns. Looks anti-aliased in the web preview but shimmers on the panel and is not locked to the refresh. Overrides `smooth_scroll` when on |
| `smooth_scroll` | bool, `true` |
| `extend_threshold_screens` | float, `2.0` |
| `auto_trim` | bool, `true` |
| `trim_threshold` | int, `10` |
@@ -180,5 +175,5 @@ See [PLUGIN_CONFIG_CORE_PROPERTIES.md](PLUGIN_CONFIG_CORE_PROPERTIES.md).
| Key | Meaning |
|---|---|
| `github.api_token` | Optional GitHub token the Plugin Store uses to avoid API rate limits (`src/plugin_system/store_registry.py`) |
| `github.api_token` | Optional GitHub token the Plugin Store uses to avoid API rate limits (`src/plugin_system/store_manager.py`) |
| `<plugin-id>.*` | Secrets a plugin declares with `"x-secret": true` in its config schema; merged into that plugin's config at load time |
+254
View File
@@ -0,0 +1,254 @@
# Creating Skins
> **Not supported yet: skins don't render with the current scoreboard
> plugins.** The only render hook is `SportsCore._render_game()` in
> `src/base_classes/sports/core.py`, and no current scoreboard (monorepo or
> third-party) builds on `src.base_classes`, so a skin you build here passes
> `validate_skin.py` but never appears on the matrix. The web UI and Plugin
> Store don't offer skins for that reason. Details:
> [SKIN_SYSTEM.md](SKIN_SYSTEM.md#status-not-supported-yet). The guide below
> stays accurate for the skin API itself.
A skin restyles a sports scoreboard (live / recent / upcoming) without
forking the plugin: the plugin keeps fetching data, scheduling, caching, and
doing vegas mode; your skin only draws. Architecture background:
[SKIN_SYSTEM.md](SKIN_SYSTEM.md).
## Quick start
```bash
cp -r skins/example-classic-baseball skins/my-skin
# edit skins/my-skin/skin.json -> set id ("my-skin"), name, author, class_name
# edit skins/my-skin/skin.py -> rename the class, start restyling
python scripts/validate_skin.py --skin my-skin
```
The validator renders your skin against bundled fixture games at several
panel sizes with **no hardware, no network, no running service**, saves PNGs
(plus 4x previews) to `skin_renders/`, and fails loudly on errors. Iterate:
edit → validate → look at the PNGs.
To select it, add to your plugin's section in `config/config.json` (this is
stored and validated, but has no visible effect until a scoreboard uses the
skin hook — see the note at the top):
```json
"baseball-scoreboard": {
"skin": "my-skin",
"skin_options": { }
}
```
The web UI's **Visual Skin** dropdown is hidden while skins are unsupported.
`"skin"` also accepts a per-mode mapping:
`{"live": "my-skin", "recent": "built-in"}`.
## The manifest (`skin.json`)
```json
{
"id": "my-skin",
"name": "My Skin",
"version": "1.0.0",
"author": "you",
"description": "What it looks like",
"skin_api_version": "1.0.0",
"targets": {
"sports": ["baseball"],
"sport_keys": ["mlb", "milb"],
"plugins": []
},
"entry_point": "skin.py",
"class_name": "MySkin",
"modes": ["live", "recent", "upcoming"],
"preview": "preview.png"
}
```
Field notes: `id` must equal the directory name; `skin_api_version`'s major
version must match the host's `SKIN_API_VERSION` or the skin is refused at
load; `targets` takes sport families (`sports`), exact sport keys
(`sport_keys`), and/or exact plugin ids (`plugins`) — any match applies.
## The renderer (`skin.py`)
```python
from src.skin_system.skin_base import ScoreboardSkin, SkinContext
class MySkin(ScoreboardSkin):
def render_live(self, ctx: SkinContext, game: dict) -> bool:
score = f"{game.get('away_score', '0')}-{game.get('home_score', '0')}"
fit = ctx.layout.fit_text(score, ctx.layout.bounds)
ctx.draw_fit(fit, ctx.layout.bounds)
return True # True = "I drew it"; False = use the built-in layout
```
Implement only the modes you care about — anything else falls back to the
plugin's built-in rendering. Return `False` to decline a specific game (e.g.
a layout that only makes sense while a game is live).
### The rules (they keep your skin from breaking the display)
1. **Draw only onto `ctx.canvas`** (via the helpers or `ctx.draw`). Never
reassign `ctx.canvas`, never touch the display or call any update method.
2. **No I/O in render paths.** No network, no file loads per frame —
`render_live` runs every display pass, and a slow render stalls the whole
matrix (the host warns at >150 ms). Use `ctx.load_logo` (cached) and
`cache_key=` for images.
3. **Derive everything from `(ctx, game)`.** Skins must be stateless: the
live/recent/upcoming modes each get their own instance.
4. **Always `.get()` optional keys.** Only the guaranteed keys below are
promised to exist.
5. **Never hardcode pixel positions for the panel.** Use `ctx.width`/
`ctx.height`, `ctx.layout` regions and `fit_text` — your skin will be run
at sizes you didn't test (64x32, 128x64, vegas cards).
6. **No third-party dependencies.** Stdlib + PIL + what `ctx` provides.
A skin that raises 3 renders in a row is disabled until the service restarts
(the built-in layout takes over), so a bug is cosmetic — but check your logs.
## SkinContext reference
| Member | What it is |
|---|---|
| `ctx.canvas` / `ctx.draw` | Fresh RGB `PIL.Image` at display size + its `ImageDraw` (raw-PIL escape hatch) |
| `ctx.width`, `ctx.height` | Canvas size — the only size truth |
| `ctx.layout` | `LayoutContext` (see [ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md)): `bounds`, `fit_text`, `fit_text_proportional`, `fit_image`, `px`, `by_tier` |
| `ctx.draw_fit(fit, box, color, align, valign)` | Draw a `fit_text` result aligned in a `Region` (handles BDF fonts) |
| `ctx.draw_text(text, x, y, color, font)` | Positioned text (handles BDF fonts) |
| `ctx.draw_image(img, box, mode, align, valign, cache_key)` | Fit + paste an image with alpha; no-ops on `None` |
| `ctx.load_logo("home" \| "away")` | Team logo as RGBA, or `None` (always handle `None`). Cached after first use; see note below |
| `ctx.draw_text_outlined(text, (x, y), font, fill, outline_color)` | The classic scorebug outlined text (TTF fonts only) |
| `ctx.fonts` | The host's font dict — keys `score`, `time`, `team`, `status`, `detail`, `rank` |
| `ctx.options` | Your user's `skin_options` from config |
| `ctx.sport`, `ctx.view_model_version`, `ctx.logger` | Context metadata + logger |
**A note on `ctx.load_logo` vs the no-I/O rule:** `load_logo` is the one
sanctioned exception. It goes through the host's logo cache — after the
first call per team it's a pure in-memory lookup. If a logo file is missing
on disk, the *first* call may download it, exactly like the built-in
renderer does for the same game (a skin is never worse than built-in here).
Always pass a stable `cache_key` when drawing it, never load image files
yourself in a render path, and always handle `None`.
The default layout idiom — carve regions, then fit text into them:
```python
from src.adaptive_layout import scoreboard_regions
regions = scoreboard_regions(ctx.layout.bounds, ctx=ctx.layout)
ctx.draw_image(ctx.load_logo("away"), regions.away_slot, cache_key=f"logo:{game.get('away_abbr')}")
ctx.draw_image(ctx.load_logo("home"), regions.home_slot, cache_key=f"logo:{game.get('home_abbr')}")
fit = ctx.layout.fit_text("3-5", regions.score_area)
ctx.draw_fit(fit, regions.score_area)
```
`Region` supports `split_h`/`split_v`/`inset`/`top_band`/`bottom_band`/
`left_col`/`right_col` for custom carves. Raw `ctx.draw.rectangle/polygon/
ellipse/...` is always available for custom marks (see the bases diamond in
the example skin).
## The game view model
Guaranteed for every sport (view model v1.0 — renaming these breaks skins and
is treated as a breaking change upstream):
| Key | Notes |
|---|---|
| `id` | Event id (string) |
| `status_text` | Display-ready status, e.g. `"Final"`, `"7:30 PM"`, `"Bot 7th"` |
| `is_live`, `is_final`, `is_upcoming`, `is_halftime` | Booleans |
| `game_date`, `game_time` | Pre-formatted local date/time strings |
| `start_time_utc` | UTC `datetime` |
| `home_abbr`, `away_abbr` | Team abbreviations (can be 2–5 chars — fit, don't assume) |
| `home_id`, `away_id` | Team ids |
| `home_score`, `away_score` | **Strings**, not ints |
| `home_record`, `away_record` | `"58-33"` or `""` (0-0 records are blanked) |
| `home_logo_path`, `away_logo_path` | Prefer `ctx.load_logo` over touching these |
Sport extras (present for that sport, still `.get()` defensively):
- **baseball**: `inning` (int), `inning_half` (`"top"`/`"bottom"`), `balls`,
`strikes`, `outs` (ints), `bases_occupied` (`[first, second, third]`
booleans), `series_summary` (str)
- **football**: `period`, `period_text`, `clock`, `home_timeouts`,
`away_timeouts`, `down_distance_text`, `down_distance_text_long`,
`is_redzone`, `possession`, `possession_indicator` (`"home"`/`"away"`),
`scoring_event`
- **basketball**: `period`, `period_text`, `clock`
- **hockey**: `period`, `period_text`, `clock`, `power_play`, `penalties`,
`home_shots`, `away_shots`
Optional everywhere (only when the user enabled the feature): `odds` (dict),
`series_summary`, rankings-related fields.
Fixture copies of these dicts live in `src/skin_system/fixtures/` — that's
exactly what the validator feeds your skin.
## Vegas mode
You get vegas support for free: vegas captures the normal display output,
which is already your skin's rendering. Optionally implement
`render_vegas_card(ctx, game)` to return a purpose-built card at
`ctx.width x ctx.height` (sizes vary — never assume 128x32).
## Building a skin with Claude Code
Skins are ideal Claude Code projects: small, isolated, and verifiable with
one command. Paste this to start:
> You are building a **display skin** for LEDMatrix — a visual overlay for a
> sports scoreboard on a small LED matrix (commonly 128x32 or 64x32 pixels).
> First read `docs/CREATING_SKINS.md` and the reference skin in
> `skins/example-classic-baseball/`.
>
> Rules:
> - Create/modify files ONLY under `skins/<my-skin-id>/`. Do NOT modify
> anything in `src/`, `scripts/`, the plugins, or any other skin.
> - Render only from the `game` dict and `ctx` helpers. No network calls, no
> per-frame file I/O, no new pip dependencies, no touching the display —
> draw onto `ctx.canvas` and return True.
> - Use `ctx.layout` regions and `fit_text` for positioning so the skin works
> at any panel size; use `.get()` for every optional game key.
> - After every change run
> `python scripts/validate_skin.py --skin <my-skin-id>` and LOOK at the
> PNGs it writes to `skin_renders/` (the `_x4.png` files are easiest to
> read). Iterate until it passes and looks right at both 128x32 and 64x32.
>
> What I want it to look like: <describe your layout — where logos, score,
> status go; colors; what shows during live vs upcoming vs final>
Tips that keep Claude (and you) out of trouble:
- One mode at a time: get `render_live` right before touching the others —
unimplemented modes automatically use the built-in look.
- Ask for edge-case renders: long team abbreviations, missing logos
(`ctx.load_logo` returning `None`), 0-0 records, extra innings/OT.
- If the render looks cramped at 64x32, ask Claude to use
`ctx.layout.by_tier(...)` to drop elements on small panels rather than
shrinking everything.
- Never let it "fix" a problem by editing `src/` — if the skin can't do
something within its directory, that's a feature request, not a workaround.
## Pre-publish checklist
- [ ] `python scripts/validate_skin.py --skin <id> --size 128x32 --size 64x32 --size 128x64` passes
- [ ] Looked at every PNG in `skin_renders/` — nothing clipped or overlapping
- [ ] Handles a missing logo (`None`) without crashing — temporarily point a
fixture's logo path at a nonexistent file to test
- [ ] Long abbreviations (`"TA&M"`, 4–5 chars) don't overflow
- [ ] No render warning above the time budget
- [ ] `skin.json`: `id` matches the directory, `version` set,
`skin_api_version` matches the host, targets correct
- [ ] `preview.png` added (grab your favorite `_x4` render)
- [ ] Tested on real hardware if you have it — a Pi is much slower than your
dev machine
Distribute by publishing the directory as a git repo (users
`git clone <repo> skins/<id>`). Registry entries with `"type": "skin"` are
hidden and refused by the Plugin Store while skins are unsupported (see
[SKIN_SYSTEM.md](SKIN_SYSTEM.md) §Distribution).
**Trust note:** a skin is Python running inside the display service — the
same trust level as a plugin. Review code before installing skins from
others.
+6 -9
View File
@@ -54,8 +54,8 @@ rows = self.layout.bounds.inset(1).split_v(3, 1, gap=1)
self.draw_fit("12:34", rows[0]) # largest crisp font that fits
self.draw_image(logo, rows[1], mode="fill_height", crop_to_ink=True)
# Weather icons: draw_weather_icon() is deprecated, removed in 3.7.0 —
# draw your own icons (the weather plugin ships WeatherIcons)
# Weather icons
display_manager.draw_weather_icon("rain", x=10, y=10, size=16)
# Scrolling state
display_manager.set_scrolling_state(True)
@@ -72,23 +72,20 @@ cache_manager.delete("key") # alias for clear_cache(key)
# Advanced caching
data = cache_manager.get_cached_data_with_strategy("key", data_type="weather")
data = cache_manager.get_background_cached_data("key", sport_key="nhl")
# Strategy
strategy = cache_manager.get_cache_strategy("weather")
interval = cache_manager.get_sport_live_interval("nhl")
```
`get_background_cached_data()` (use `get()`) and `get_sport_live_interval()`
are deprecated, removed in 3.7.0. See
[Deprecated APIs](PLUGIN_API_REFERENCE.md#deprecated-apis).
## Plugin Manager Quick Methods
```python
# Get plugins
plugin = plugin_manager.get_plugin("plugin-id")
all_plugins = plugin_manager.get_all_plugins()
# get_enabled_plugins() is deprecated, removed in 3.7.0 — check `enabled`
# on the entries in plugin_manager.plugins
enabled = plugin_manager.get_enabled_plugins()
# Get info
info = plugin_manager.get_plugin_info("plugin-id")
@@ -171,7 +168,7 @@ def display(self, force_clear=False):
- [ ] Plugin inherits from `BasePlugin`
- [ ] Implements `update()` and `display()` methods
- [ ] `manifest.json` with the [required fields](PLUGIN_API_REFERENCE.md#manifest-required-fields)
- [ ] `manifest.json` with required fields
- [ ] `config_schema.json` for web UI (recommended)
- [ ] `README.md` with documentation
- [ ] Error handling implemented
+6 -6
View File
@@ -17,13 +17,13 @@ The LEDMatrix emulator allows you to run and test LEDMatrix displays on your com
## Prerequisites
### System Requirements
- Python 3.10 or higher
- Python 3.7 or higher
- Windows, macOS, or Linux
- At least 2GB RAM (4GB recommended)
- Internet connection for plugin downloads
### Required Software
- Python 3.10+
- Python 3.7+
- pip (Python package manager)
- Git (for plugin management)
@@ -50,7 +50,8 @@ pip install -r requirements-emulator.txt
```
This installs:
- `RGBMatrixEmulator` - the emulation library (and whatever it depends on)
- `RGBMatrixEmulator` - The core emulation library
- Additional dependencies for display adapters
### 3. Install Standard Dependencies
@@ -62,9 +63,8 @@ pip install -r requirements.txt
### 1. Emulator Configuration File
The emulator uses `emulator_config.json` for configuration. It isn't in
the repo (it's gitignored): RGBMatrixEmulator writes it on first run.
A typical file looks like this:
The emulator uses `emulator_config.json` for configuration. Here's the
default configuration as it ships in the repo:
```json
{
+327 -122
View File
@@ -9,14 +9,12 @@
## Overview
[`src/font_manager.py`](../src/font_manager.py) loads and caches the TTF and
BDF fonts in `assets/fonts/`, registers fonts that plugins ship, and records
which plugin uses which font so the web UI can show it.
Several methods are deprecated and will be removed in LEDMatrix 3.7.0; they
log a warning on first call. They are listed in
[Deprecated methods](#deprecated-methods) below, and the full set is pinned in
[`test/test_deprecation.py`](../test/test_deprecation.py).
The enhanced FontManager provides comprehensive font management for the LEDMatrix application with support for:
- Manager font registration and detection
- Plugin font management
- Programmatic per-element font overrides
- Performance monitoring and caching
- Dynamic font discovery
## Getting the FontManager
@@ -36,60 +34,157 @@ standalone FontManager when none is available (test harnesses, mocks).
`DisplayManager` has **no** `font_manager` attribute —
`display_manager.font_manager` raises `AttributeError`.
## Resolving a font
## Architecture
```python
element_key = f"{self.plugin_id}.title"
### Manager-Centric Design
# Register the choice so the web UI's Fonts tab can list it.
self.font_manager.register_manager_font(
manager_id=self.plugin_id,
element_key=element_key,
family="press_start",
size_px=10,
color=(255, 255, 255),
)
Managers define their own fonts, but the FontManager:
1. **Loads and caches fonts** for performance
2. **Detects font usage** for visibility
3. **Allows manual overrides** when needed
4. **Supports plugin fonts** with namespacing
font = self.font_manager.resolve_font(
element_key=element_key,
family="press_start",
size_px=10,
)
self.display_manager.draw_text("Hello", x=10, y=10, font=font)
### Font Resolution Flow
```
Manager requests font → Check manual overrides → Apply manager choice → Cache & return
```
`resolve_font()` applies any entry for `element_key` in
`config/font_overrides.json`, maps a plugin-local family to its namespaced
name when `plugin_id` is passed, and then calls `get_font(family, size_px)`.
On error it returns a fallback font rather than raising.
## For Manager Developers
`get_font(family, size_px)` looks the family up in `font_catalog` and loads
it (cached per family and size).
### Basic Font Usage
## Font families
```python
from src.font_manager import FontManager
At start-up the FontManager scans `assets/fonts/` for `.ttf` and `.bdf`
files. Each becomes a family named after the file, lower-cased and without
the extension (`PressStart2P-Regular.ttf` → `pressstart2p-regular`). Four
aliases are added on top:
class MyManager:
def __init__(self, config, display_manager, cache_manager, plugin_manager):
self.display_manager = display_manager
self.font_manager = plugin_manager.font_manager # Shared FontManager
self.manager_id = "my_manager"
def display(self):
# Define your font choices
element_key = "my_manager.title"
font_family = "press_start"
font_size_px = 10
color = (255, 255, 255) # RGB white
# Register your font choice (for detection and future overrides)
self.font_manager.register_manager_font(
manager_id=self.manager_id,
element_key=element_key,
family=font_family,
size_px=font_size_px,
color=color
)
# Get the font (checks for manual overrides automatically)
font = self.font_manager.resolve_font(
element_key=element_key,
family=font_family,
size_px=font_size_px
)
# Use the font for rendering
self.display_manager.draw_text(
"Hello World",
x=10, y=10,
color=color,
font=font
)
```
| Alias | File |
|---|---|
| `press_start` | `assets/fonts/PressStart2P-Regular.ttf` |
| `four_by_six` | `assets/fonts/4x6-font.ttf` |
| `five_by_seven` | `assets/fonts/5x7.bdf` |
| `tom_thumb` | `assets/fonts/tom-thumb.bdf` |
### Advanced Font Usage
Read the catalog directly: `font_manager.font_catalog` is a dict of family
name to file path. Files added later are picked up on the next start of the
display service.
```python
class AdvancedManager:
def __init__(self, config, display_manager, cache_manager, plugin_manager):
self.display_manager = display_manager
self.font_manager = plugin_manager.font_manager
self.manager_id = "advanced_manager"
# Define your font specifications
self.font_specs = {
"title": {"family": "press_start", "size_px": 12, "color": (255, 255, 0)},
"body": {"family": "four_by_six", "size_px": 8, "color": (255, 255, 255)},
"footer": {"family": "five_by_seven", "size_px": 7, "color": (128, 128, 128)}
}
# Register all font specs
for element_type, spec in self.font_specs.items():
element_key = f"{self.manager_id}.{element_type}"
self.font_manager.register_manager_font(
manager_id=self.manager_id,
element_key=element_key,
family=spec["family"],
size_px=spec["size_px"],
color=spec["color"]
)
def get_font(self, element_type: str):
"""Helper method to get fonts with override support."""
spec = self.font_specs[element_type]
element_key = f"{self.manager_id}.{element_type}"
return self.font_manager.resolve_font(
element_key=element_key,
family=spec["family"],
size_px=spec["size_px"]
)
def display(self):
# Get fonts (automatically checks for overrides)
title_font = self.get_font("title")
body_font = self.get_font("body")
footer_font = self.get_font("footer")
# Render with fonts
self.display_manager.draw_text("Title", font=title_font, color=self.font_specs["title"]["color"])
self.display_manager.draw_text("Body Text", font=body_font, color=self.font_specs["body"]["color"])
self.display_manager.draw_text("Footer", font=footer_font, color=self.font_specs["footer"]["color"])
```
## Plugin fonts
### Using Size Tokens
Plugins that ship their own fonts declare them in a `"fonts"` block in
`manifest.json`. The plugin manager calls
`FontManager.register_plugin_fonts()` during plugin load. `plugin://…`
sources are resolved relative to the plugin's install directory.
```python
# Get available size tokens
tokens = self.font_manager.get_size_tokens()
# Returns: {'xs': 6, 'sm': 8, 'md': 10, 'lg': 12, 'xl': 14, 'xxl': 16}
# Use token to get size
size_px = tokens.get('md', 10) # 10px
# Then use in font resolution
font = self.font_manager.resolve_font(
element_key="my_manager.text",
family="press_start",
size_px=size_px
)
```
## For Plugin Developers
> **Note**: plugins that ship their own fonts via a `"fonts"` block
> in `manifest.json` are registered automatically during plugin load
> (`src/plugin_system/plugin_manager.py` calls
> `FontManager.register_plugin_fonts()`). The `plugin://…` source
> URIs documented below are resolved relative to the plugin's
> install directory.
>
> The web UI's **Fonts** tab lists, uploads, previews and deletes the
> font files in `assets/fonts/`. Its **Used by** column shows which
> loaded plugins registered each file through `register_manager_font()`
> (see [Font usage in the web UI](#font-usage-in-the-web-ui)), and it
> warns before deleting one of them. It has no override editor (the
> override panels and `/api/v3/fonts/overrides` endpoints were removed).
> The programmatic override workflow in
> [Manual Font Overrides](#manual-font-overrides) below still works.
> Let users pick fonts through your plugin's own config schema.
### Plugin Font Registration
In your plugin's `manifest.json`:
```json
{
@@ -100,123 +195,233 @@ sources are resolved relative to the plugin's install directory.
{
"family": "custom_font",
"source": "plugin://fonts/custom.ttf",
"metadata": {"description": "Custom plugin font", "license": "MIT"}
"metadata": {
"description": "Custom plugin font",
"license": "MIT"
}
},
{
"family": "web_font",
"source": "https://example.com/fonts/font.ttf",
"metadata": {"checksum": "sha256:abc123..."}
"metadata": {
"description": "Downloaded font",
"checksum": "sha256:abc123..."
}
}
]
}
}
```
Registered families are namespaced as `<plugin_id>::<family>`. Pass
`plugin_id` to `resolve_font()` to use the short name:
### Using Plugin Fonts
```python
font = self.font_manager.resolve_font(
element_key=f"{self.plugin_id}.text",
family="custom_font", # resolved as "my-plugin::custom_font"
size_px=10,
plugin_id=self.plugin_id,
)
class MyPlugin(BasePlugin):
def __init__(self, plugin_id, config, display_manager, cache_manager, plugin_manager):
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
self.font_manager = self._get_font_manager()
def display(self):
# Use plugin font (automatically namespaced)
font = self.font_manager.resolve_font(
element_key=f"{self.plugin_id}.text",
family="custom_font", # Will be resolved as "my-plugin::custom_font"
size_px=10,
plugin_id=self.plugin_id
)
self.display_manager.draw_text("Plugin Text", font=font)
```
## Overrides
## Manual Font Overrides
`resolve_font()` still honours `config/font_overrides.json` (a map of
element key to `family` and/or `size_px`), which is read once at start-up.
The methods that edit it — `set_override()`, `remove_override()`,
`get_overrides()` — are deprecated, and there is no web UI or REST endpoint
for overrides (the override editor and `/api/v3/fonts/overrides` were
removed). To let users choose a font, add a field to your plugin's config
schema.
Overrides are set in code (there is no web UI or REST endpoint for them).
They are stored in `config/font_overrides.json` and persist across restarts.
### Programmatic Overrides
```python
# Set override
font_manager.set_override(
element_key="nfl.live.score",
family="four_by_six",
size_px=8
)
# Remove override
font_manager.remove_override("nfl.live.score")
# Get all overrides
overrides = font_manager.get_overrides()
```
## Font Discovery
### Available Fonts
The FontManager automatically scans `assets/fonts/` for TTF and BDF fonts:
```python
# Get all available fonts
fonts = font_manager.get_available_fonts()
# Returns: {'press_start': 'assets/fonts/PressStart2P-Regular.ttf', ...}
# Check if font exists
if "my_font" in fonts:
font = font_manager.get_font("my_font", 10)
```
### Adding Custom Fonts
Place font files in `assets/fonts/` directory:
- Supported formats: `.ttf`, `.bdf`
- Font family name is derived from filename (without extension)
- Will be automatically discovered on next initialization
## Font usage in the web UI
The web UI's **Fonts** tab lists, uploads, previews and deletes the font
files in `assets/fonts/`. The web interface runs in its own process and has
no FontManager, so the display service publishes which plugin uses which
font ([`src/font_usage.py`](../src/font_usage.py)), and the tab's **Used by**
column reads it:
The web interface runs in its own process and has no FontManager, so the
display service publishes which plugin uses which font
(`src/font_usage.py`), and the Fonts tab's **Used by** column reads it:
- **Source**: `register_manager_font()` registrations of the loaded
plugins. `get_font()` and `resolve_font()` do not know the calling plugin
and are not counted, and neither is a plugin that opens a font file
directly with PIL — register the fonts your plugin draws with if you want
them listed.
- **Names**: a family, alias or path is resolved through `font_catalog` to
the file it loads and reported under that file's name without extension
(`PressStart2P-Regular`, `4x6-font`, `5x7`, `tom-thumb`), which is how the
Fonts tab keys its rows. Fonts outside `assets/fonts/` (a plugin's own
`plugin_id::family` fonts) and families that resolve to nothing are left
out.
- **Names**: a family, alias (`press_start`, `four_by_six`,
`five_by_seven`, `tom_thumb`) or path is resolved through
`font_catalog` to the file it loads and reported under that file's name
without extension (`PressStart2P-Regular`, `4x6-font`, `5x7`,
`tom-thumb`), which is how the Fonts tab keys its rows. Fonts outside
`assets/fonts/` (a plugin's own `plugin_id::family` fonts) and families
that resolve to nothing are left out.
- **When**: a daemon thread started once plugins have loaded checks every
10 seconds and writes the `font_usage_snapshot` cache key only when the
usage changed (and once a day, so the cache's cleanup never expires it).
Unloading a plugin drops its registrations (`forget_manager_fonts`).
- **Unknown**: until the display service has published, the column reads
"unknown" and `GET /api/v3/fonts/catalog` returns `used_by: null`.
- The tab warns before deleting a font that a loaded plugin registered.
## Text measurement
## Performance Monitoring
```python
# Get performance stats
stats = font_manager.get_performance_stats()
print(f"Cache hit rate: {stats['cache_hit_rate']*100:.1f}%")
print(f"Total fonts cached: {stats['total_fonts_cached']}")
print(f"Failed loads: {stats['failed_loads']}")
print(f"Manager fonts: {stats['manager_fonts']}")
print(f"Plugin fonts: {stats['plugin_fonts']}")
```
## Text Measurement
```python
# Measure text dimensions
width, height, baseline = font_manager.measure_text("Hello", font)
# Get font height
font_height = font_manager.get_font_height(font)
```
## Tips
## Best Practices
- BDF fonts usually look better than TTF at small sizes on LED panels.
- Use `{plugin_id}.{element}` element keys.
- Register the fonts you draw with, so the Fonts tab can warn before one is
deleted.
- Replace direct `ImageFont.truetype("assets/fonts/...", 8)` calls with
`resolve_font()`: it caches, resolves paths against the install directory,
and handles BDF files.
### For Managers
1. **Register all fonts** you use for visibility
2. **Use consistent element keys** (e.g., `{manager_id}.{element_type}`)
3. **Cache font references** if using same font multiple times
4. **Use `resolve_font()`** not `get_font()` directly to support overrides
5. **Define sensible defaults** that work well on LED matrix
### For Plugins
1. **Use plugin-relative paths** (`plugin://fonts/...`)
2. **Include font metadata** (license, description)
3. **Provide fallback** fonts if custom fonts fail to load
4. **Test with different display sizes**
### General
1. **BDF fonts** are often better for small sizes on LED matrices
2. **TTF fonts** work well for larger sizes
3. **Monospace fonts** are easier to align
4. **Test on actual hardware** - what looks good on screen may not work on LED matrix
## Migration from Old System
### Old Way (Direct Font Loading)
```python
self.font = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 8)
```
### New Way (FontManager)
```python
element_key = f"{self.manager_id}.text"
self.font_manager.register_manager_font(
manager_id=self.manager_id,
element_key=element_key,
family="pressstart2p-regular",
size_px=8
)
self.font = self.font_manager.resolve_font(
element_key=element_key,
family="pressstart2p-regular",
size_px=8
)
```
## Troubleshooting
**Font not found**
- Check the file exists in `assets/fonts/`.
- The family name is the filename without extension, lower-cased.
- Check the display service log for font discovery errors.
### Font Not Found
- Check font file exists in `assets/fonts/`
- Verify font family name matches filename (without extension, lowercase)
- Check logs for font discovery errors
**Plugin fonts not loading**
- Check the manifest's `"fonts"` block.
- Check the log for download or registration errors, and that font URLs are
reachable.
### Override Not Working
- Verify element key matches exactly what manager registered
- Check `config/font_overrides.json` for correct syntax
- Restart application to ensure overrides are loaded
## API reference
### Performance Issues
- Check cache hit rate in performance stats
- Reduce number of unique font/size combinations
- Clear cache if it grows too large: `font_manager.clear_cache()`
Current methods:
### Plugin Fonts Not Loading
- Verify plugin manifest syntax
- Check plugin directory structure
- Review logs for download/registration errors
- Ensure font URLs are accessible
| Method | Purpose |
|---|---|
| `register_manager_font(manager_id, element_key, family, size_px, color=None)` | Record a font choice (feeds the Fonts tab) |
| `forget_manager_fonts(manager_id)` | Drop a manager's registrations (core calls it when a plugin unloads) |
| `resolve_font(element_key, family, size_px, plugin_id=None)` | Get a font, applying overrides and plugin namespacing |
| `get_font(family, size_px)` | Get a font directly |
| `get_native_bdf_size(family)` | Native pixel size of a BDF family, or `None` |
| `measure_text(text, font)` | `(width, height, baseline)` |
| `get_font_height(font)` | Line height |
| `register_plugin_fonts(plugin_id, font_manifest)` | Register a plugin's fonts (core calls it at load) |
| `clear_cache()` | Drop cached fonts and metrics |
| `font_catalog` (attribute) | Family name → file path |
## API Reference
### Deprecated methods
### FontManager Methods
Removed in 3.7.0. Each logs a warning on first call.
- `register_manager_font(manager_id, element_key, family, size_px, color=None)` - Register font usage
- `forget_manager_fonts(manager_id)` - Drop a manager's registrations (core calls it when a plugin unloads)
- `resolve_font(element_key, family, size_px, plugin_id=None)` - Get font with override support
- `get_font(family, size_px)` - Get font directly (bypasses overrides)
- `measure_text(text, font)` - Measure text dimensions
- `get_font_height(font)` - Get font height
- `set_override(element_key, family=None, size_px=None)` - Set manual override
- `remove_override(element_key)` - Remove override
- `get_overrides()` - Get all overrides
- `get_detected_fonts()` - Get all detected font usage
- `get_manager_fonts(manager_id=None)` - Get fonts by manager
- `get_available_fonts()` - Get font catalog
- `get_size_tokens()` - Get size token definitions
- `get_performance_stats()` - Get performance metrics
- `clear_cache()` - Clear font cache
- `register_plugin_fonts(plugin_id, font_manifest)` - Register plugin fonts
- `unregister_plugin_fonts(plugin_id)` - Unregister plugin fonts
## Example: Complete Manager Implementation
For a working example of the font manager API in use, see
`src/font_manager.py` itself and the bundled scoreboard base classes
in `src/base_classes/` (e.g., `hockey.py`, `football.py`) which
register and resolve fonts via the patterns documented above.
| Method | Use instead |
|---|---|
| `get_available_fonts()`, `get_font_catalog()` | read `font_catalog` |
| `get_size_tokens()` | pass a pixel size |
| `get_performance_stats()` | — |
| `set_override()`, `remove_override()`, `get_overrides()` | a font field in your plugin's config schema |
| `get_manager_fonts()`, `get_detected_fonts()` | — |
| `get_plugin_fonts()`, `unregister_plugin_fonts()` | — |
| `add_font()`, `remove_font()`, `validate_font()` | the web UI's Fonts tab |
+10 -11
View File
@@ -116,8 +116,8 @@ weather and other location-aware plugins.
4. Wait for installation to finish — installed plugins appear in the
**Installed Plugins** section above and get their own tab in the second
nav row
5. Toggle the plugin to enabled. The running display loads it within a
few seconds; no restart is needed
5. Toggle the plugin to enabled
6. From **Overview**, click **Restart Display Service**
You can also install community plugins straight from a GitHub URL using the
**Install from GitHub** section further down the same tab — see
@@ -128,9 +128,9 @@ You can also install community plugins straight from a GitHub URL using the
1. Each installed plugin gets its own tab in the second navigation row
2. Open that plugin's tab to edit its settings (favorite teams, API keys,
update intervals, etc.)
3. Click **Save**. The display service watches `config.json` and hands the
new settings to the running plugin, so no restart is needed. If a plugin
still shows old settings, restart the display service from **Overview**
3. Click **Save**
4. Restart the display service from **Overview** so the new settings take
effect
**Note:** how long each plugin stays on screen is not set in the
plugin's own tab — use the **Rotation** tab's **Screen Durations**
@@ -197,15 +197,14 @@ The fastest way to verify a plugin works without waiting for the rotation:
**Check:**
1. Plugin is enabled (toggle on the **Plugin Manager** tab)
2. Plugin's display duration is non-zero
3. No errors in the **Logs** tab for that plugin. A plugin whose
`validate_config()` fails is not loaded until its settings are fixed
2. Display service was restarted after enabling
3. Plugin's display duration is non-zero
4. No errors in the **Logs** tab for that plugin
**Fix:**
1. Enable the plugin from **Plugin Manager**
2. Check the **Logs** tab for plugin-specific errors
3. If it still does not appear, click **Restart Display Service** on
**Overview**
2. Click **Restart Display Service** on **Overview**
3. Check the **Logs** tab for plugin-specific errors
### Weather Plugin Shows "No Data"
+5 -5
View File
@@ -52,10 +52,10 @@ pytest test/test_display_controller.py test/test_plugin_system.py
```bash
# Run a specific test class
pytest test/test_display_controller.py::TestDisplayControllerLivePriority
pytest test/test_display_controller.py::TestDisplayControllerModeRotation
# Run a specific test function
pytest test/test_display_controller.py::TestDisplayControllerSchedule::test_active_hours
pytest test/test_display_controller.py::TestDisplayControllerModeRotation::test_basic_rotation
```
### Run Tests by Marker
@@ -98,7 +98,7 @@ When you run `pytest`, you'll see:
```
test/test_display_controller.py::TestDisplayControllerInitialization::test_init_success PASSED
test/test_display_controller.py::TestDisplayControllerOnDemand::test_activate_on_demand PASSED
test/test_display_controller.py::TestDisplayControllerModeRotation::test_basic_rotation PASSED
...
```
@@ -174,10 +174,10 @@ pytest
```bash
# Run with maximum verbosity and show print statements
pytest -vv -s test/test_display_controller.py::TestDisplayControllerSchedule::test_active_hours
pytest -vv -s test/test_display_controller.py::TestDisplayControllerModeRotation::test_basic_rotation
# Run with Python debugger (pdb)
pytest --pdb test/test_display_controller.py::TestDisplayControllerSchedule::test_active_hours
pytest --pdb test/test_display_controller.py::TestDisplayControllerModeRotation::test_basic_rotation
```
### Run Tests in Parallel (Faster)
-388
View File
@@ -1,388 +0,0 @@
# Offscreen Rendering
**Status (2026-09-24):** step 1, offscreen rendering, is implemented
(`DisplayManager.offscreen()`, the adapter on the prefetch thread, the plugin
lock). Steps 2 and 3 are proposed. When all three land, this file becomes the
reference for how plugin content is rendered off the render thread.
First soak of step 1 on hdpi (50 px/s, `pwm_bits` 7, preview open, 8-minute
runs, A/B/B/A):
| build | late | by 1 | 2 | 3–5 | 6+ | freezes | render-thread fetches |
|---|---|---|---|---|---|---|---|
| #628 | 0.53% | 82 | 2 | 3 | 2 | 3 | 6 |
| step 1 | 0.63% | 78 | 63 | 17 | 2 | 1 | 0 |
| step 1 | 0.42% | 77 | 23 | 10 | 0 | 0 | 0 |
| #628 | 0.37% | 84 | 5 | 3 | 3 | 2 | 14 |
It does what it was built to: no plugin is fetched on the render thread, and
freezes fell from 5 to 1. But frames 2–5 refreshes late rose. The rendering
moved to the prefetch thread still needs the GIL, and the render thread waits
for it (risk 5 below). The late rate did not improve overall. The 1–2 s
freezes appear in both builds and have a separate, not yet identified cause.
The GIL fix, measured on hdpi (90 px/s, `pwm_bits` 8, preview open, 8-minute
runs after a 2-minute warm-up, order A B C C B A, 2026-09-24). Each arm pools
two runs, about 81,000 frames:
| arm | late | by 1 | 2 | 3–5 | 6+ | 2+ late per 10k frames | freezes |
|---|---|---|---|---|---|---|---|
| A: step 1 as is | 0.90% | 575 | 64 | 91 | 9 | 20.1 | 0 |
| B: `switch_interval_ms` 1 | 0.78% | 510 | 105 | 23 | 2 | 15.8 | 0 |
| C: `prefetch_gate` | **0.60%** | 471 | 11 | 7 | 2 | **2.5** | 0 |
The gate removes the frames the render thread spent waiting for the GIL, and
it costs the prefetch nothing that shows: it parked the thread for 3–6 s per
run, and the next group was ready at every strip extension in every arm.
`prefetch_gate` is therefore on by default; `switch_interval_ms` stays an
off-by-default experiment. What is left is almost all one refresh late, which
is the per-frame budget (a 6.75 ms p50 blit in a refresh the panel holds at
83–85 Hz while rendering), not contention.
The runs restart the service, so the hourly sports refresh never fell inside
one. That refresh is its own case: about twenty ESPN chunk-fetch threads at
once, which the gate does not cover (it gates only the prefetch thread).
## The problem
Vegas mode builds its ticker from every plugin's content. Most of that work
already happens on a background prefetch thread
(`RenderPipeline.start_prefetch`). But any plugin whose content needs the
**shared display canvas** is deferred to the render thread
(`RenderPipeline.drain_deferred`), one plugin every two seconds. The code's
own comments put each of those at 40–600 ms, and the render thread presents no
frames while one runs.
On hdpi (Pi 4, 512×64) most plugins take that path: geochron, tide-display,
news, hockey-scoreboard, ledmatrix-stocks, incoming-packages, clock-simple,
countdown, birdnet-go, ledmatrix-music and odds-ticker. They arrive in bursts
("Whole group deferred; strip will extend as it drains") every minute or so,
12 fetches in five minutes. That is the "occasional pause" a viewer sees.
An 8-minute soak (`scripts/frame_soak.py --preview`) of the #628 build on
hdpi:
| late by | frames |
|---|---|
| 1 refresh | 238 |
| 2 | 32 |
| 3–5 | 30 |
| 6+ | 5 |
| freezes ≥ 250 ms | 2 (0.97 s total) |
The 3+ rows and the freezes are the pauses. The single-refresh row is a
separate problem: the blit is 6 ms of a 10 ms refresh, so there is little
slack. It is covered under *What this does not fix*.
## Why a plugin is canvas-bound
The plugin-facing canvas is a set of shared attributes on `DisplayManager`:
`image`, `draw`, `matrix`, and the `width`/`height` properties that read from
`matrix`. Three adapter paths (`src/vegas_mode/plugin_adapter.py`) need them,
and each returns `None` under `offscreen_only=True` so the plugin is queued for
the render thread:
1. **Display capture** (`_capture_display_content`): clear the canvas, call
`plugin.display()`, copy `display_manager.image`. Used by any plugin
without `get_vegas_content()` or a populated `scroll_helper`.
2. **Scroll-content generation** (`_trigger_scroll_content_generation`): a
ticker plugin whose `scroll_helper.cached_image` is empty is made to build
it by calling `display(force_clear=True)` or `_create_scrolling_display()`.
Both draw on the canvas.
3. **Narrowed rendering** (`DisplayManager.render_size`): swaps the shared
`matrix`, `image` and `draw` for a narrower set so the plugin lays out for
`render_width_pct`. The render thread would see the swap mid-frame.
The render thread keeps the canvas coherent only because nothing else touches
it at the same time. A background thread can't use it.
## The design: a per-thread render target
`capture_mode()` is already per-thread (#423 made its state a
`threading.local`, so a background capture no longer suppresses the render
loop's pushes). The same move applies to the canvas itself:
```python
with display_manager.offscreen(width=None, height=None) as surface:
plugin.display(force_clear=True)
content = surface.image.copy()
```
For the **calling thread only**, inside the block:
| accessor | resolves to |
|---|---|
| `display_manager.image`, `.draw` | the surface's own image and draw: a fresh black canvas, `fontmode = "1"` |
| `display_manager.matrix` | a logical proxy reporting the surface size, so `width`/`height` and plugins that read `matrix.width` follow it. Hardware calls through it (`SetImage`, `SwapOnVSync`, `Clear`, brightness writes) are inert. |
| `update_display()`, `clear()` | canvas-only: the block implies capture mode, which is already per-thread |
| `set_scrolling_state()`, `set_frame_hold()` | no-ops, so a plugin's `display()` cannot re-pace the live scroll. Today it can, when it is captured on the render thread. |
Every other thread sees the real canvas, unchanged. The render loop in
particular keeps presenting while a plugin draws elsewhere.
### Implementation sketch
- `image`, `draw` and `matrix` become properties over `_image`, `_draw` and
`_matrix`, plus a thread-local current surface. The getter returns the
surface's value when the calling thread has one, else the shared one; setters
mirror that. That costs about 0.1 µs per access, and `update_display()` reads
each a handful of times per frame. Every existing `self.image = ...` in
`DisplayManager` (`clear()`, setup, fallback) keeps working and becomes
thread-correct for free.
- `render_size()` is rebuilt on `offscreen()`: it creates or narrows the
calling thread's surface instead of swapping shared state.
- `offscreen()` nests and always restores on exit, including when the plugin
raises.
- `VisualDisplayManager` (the plugin test harness) gets the same method, for
parity.
### Adapter changes
- `get_content(offscreen_only=True)` stops returning `None` for the three
paths above. Each runs inside `display_manager.offscreen(render_width)`.
- `_capture_display_content` and `_trigger_scroll_content_generation` drop
their "copy the shared image, restore it afterwards" bookkeeping, since the
shared image is never touched.
- **Take the plugin's lock.** `PluginManager.get_plugin_lock()` keeps
`update()` and `display()` mutually exclusive in normal rotation, but Vegas
never takes it, so today's render-thread captures already race
`update()`. Off the render thread the adapter can afford to wait: blocking
acquire with a timeout (proposed 2 s). On timeout it keeps the cached segment
and tries again next group.
- `drain_deferred()` and the deferred queue are deleted. The only render-thread
fetch left is the inline fallback when no prepared group is ready, which in
practice is the first extension. Prefetching at start removes that too.
## Keeping live content fresh
Offscreen rendering is also what makes fresh sports scores possible. Today a
plugin's segment is drawn when its group is prefetched, and the strip carries
7,000–10,000 px of content ahead of the viewport (hdpi logs: "7153px still
ahead", "9842px ahead"). At ~100 px/s, a score drawn now reaches the screen
70–100 seconds later. When a plugin reports new data, Vegas only drops its
cache (`invalidate_pending_updates`), so the change is drawn on the plugin's
*next* turn, several minutes later. A segment already in the strip scrolls by
with the data it was drawn with.
That was the right trade while every redraw of a canvas-bound plugin stalled
the scroll. Off the render thread a redraw costs the scroll nothing, so the
strip can afford three things.
### 1. Refresh at the gate
Before a segment enters the viewport, check whether its plugin has updated
since the segment was drawn. If it has, redraw it offscreen and replace it
while it is still out of sight. Width changes are fine here, because
everything from that segment onward is still invisible.
The gate sits `lead` pixels ahead of the viewport's right edge:
`lead = max(one screen, speed × (render time + margin))`. The render time is
the plugin's own, measured on each render (sports cards take the longest,
hundreds of ms up to seconds per the prefetch notes). A plugin whose render
does not finish before its segment reaches the viewport keeps the old segment.
The scroll never waits for it.
Content is then at most `lead / speed` seconds old when it appears, a few
seconds instead of minutes, without changing how far ahead the rotation
fetches.
### 2. Replace ahead of the screen
When a plugin reports new data (the Vegas update tick already names them), any
of its segments that are **anywhere ahead of the viewport** are redrawn and
replaced straight away, not only at the gate. That covers the long stretch of
strip between prefetch and the gate.
### 3. Update on screen
A segment that is already **visible** is patched in place when the redrawn
version has the same geometry: the same total width, and the same width for
each card (a sports plugin returns one image per game, joined with
`intra_plugin_gap`). Scoreboard cards keep a fixed layout, so a score change
patches in and the digits update as the card scrolls past. The patch is a
pixel copy of one card (a 150×64 card is ~29 KB) applied by the render thread
between frames, so a frame never shows half of a patch.
When the geometry differs (a game added or dropped, a card that grew), the
visible part cannot change without a jump. Only the cards not yet on screen
are replaced, and only if the geometry up to that point is unchanged. Otherwise
the segment keeps its snapshot until it has scrolled off.
### Avoiding wasted work
- **Change detection.** `run_scheduled_updates_with_changes()` names a plugin
whenever its `update()` ran, not when its data changed. On hdpi
`clock-simple` and `ledmatrix-music` are named on every 4-second tick. A
redraw whose pixels hash the same as the segment's is discarded without a
swap.
- **Redraw on real updates only.** Vegas makes no API calls. Each plugin
fetches on its own schedule, and a redraw is triggered only when the
plugin's `update()` has run since its segment was drawn. On hdpi live
football, baseball and hockey poll every 30 s (live odds every 60 s,
everything else hourly), so a live sports card is redrawn once per poll.
- **Floor.** A plugin is redrawn at most once per
`vegas_scroll.refresh_min_interval` (proposed 10 s), and never while its
previous redraw is still running. The floor never holds back a sports card
polling every 30 s. It exists for chatty plugins: `clock-simple` updates
every second and `ledmatrix-music` polls every 2 s.
- **One worker.** Redraws go through the same background worker as prefetch,
one plugin at a time at `nice 10`, under the plugin's lock.
Data freshness is still bounded by each plugin's own fetch interval (how often
it polls live scores). Drawing faster cannot beat the data source.
### The strip becomes a list of segments
All three need the strip to be replaceable by segment. Today it is one
image (`ScrollHelper.cached_array`, 8,000–20,000 px wide, 1.5–3.8 MB), and
`append_content()` rebuilds the whole thing on the render thread for every
appended block. That is also a pause source.
Proposed `SegmentStrip`, used by Vegas in place of the single image:
- an ordered list of segments: plugin id, card boundaries, a pixel array, the
render time, and the plugin data version it was drawn from, plus its
x-offset in the strip;
- `visible(x, width)` assembles the viewport by slicing across at most a few
segments: the same ~100 KB copy per frame that slicing the single image
costs today;
- append and trim become O(block) list operations, not a copy of the strip;
- replace swaps one list entry and shifts the offsets of the segments after it
(dozens at most). A same-geometry patch copies pixels into the existing array.
Every mutation is prepared off the render thread and applied by the render
thread at a frame boundary, so the strip the render loop reads is never
half-changed.
### Multi-display sync
The follower renders from its own copy of the strip, offset from the leader's
scroll position. Today the leader sends that copy whole, and only in
`start_new_cycle()` (`send_scroll_image`), plus the scroll position every
frame. Continuous scroll, the default, extends and trims the strip without
starting a new cycle, and nothing sends those changes. From reading the code,
the follower therefore probably falls out of step after the first extension
already, before any of this design. That is untested; it needs a two-Pi rig.
With a segment strip, keeping the follower identical becomes **replaying the
leader's operations**:
- Every strip mutation (append, trim, replace, patch) is one operation in
strip coordinates. The leader applies it and sends the same operation to the
follower over the existing TCP channel. Segments are small: a card is ~29 KB
raw and compresses well.
- Operations on off-screen segments apply on arrival. A patch to a segment
that is on either panel carries an *apply at scroll position X* stamp a
couple of hundred milliseconds ahead. Both sides apply it when their scroll
position passes X, so both panels change on the same frame, within the
existing position-sync jitter.
- Each operation carries a sequence number. A follower that sees a gap (a
reconnect, a dropped message) asks for a full snapshot, which is today's
`send_scroll_image` path.
That also fixes the probable continuous-mode gap as a side effect, since
appends and trims become operations too. Until it is in place, fresh-content
updates are disabled while sync is active.
## Risks, and what was checked
1. **Plugins holding their own reference to the shared `draw` or `image`.**
They would keep drawing into the shared canvas, and routing by thread can't
redirect them. A grep of the 49 plugins installed on hdpi found none storing
`display_manager.draw` or `.image` in an attribute (a pattern search, so
indirect aliasing would slip past it). A plugin that did would
draw into an image nobody displays, which trims to a blank segment. That is
not corruption, and it is no worse than today.
2. **Plugins calling the matrix directly.** None in the audit. Inside
`offscreen()` the proxy makes it inert anyway.
3. **Font thread-safety.** `FontManager` shares font objects across plugins.
Measured on Pillow 12.3, two threads rendering text take 1.94× as long as
one, so text rendering holds the GIL and FreeType is never entered
concurrently. Re-check if Pillow changes that.
4. **Plugin thread-safety.** `display()` moves to the prefetch thread. The
plugin lock makes it exclusive with `update()`, which is more protection
than it has today. Threads a plugin starts itself are not covered, as today.
5. **The GIL.** Moving 40–600 ms of plugin rendering off the render thread
removes the pauses, but the work still needs the GIL. Pillow drawing holds
it, and a waiting thread only gets it back after the switch interval
(default 5 ms). Expect some single-refresh late frames while a prefetch
runs. Measure with the soak. A render process separate from plugin work
is the structural answer (the "native presenter" step). Two experiments
get most of the way first (results under Status, above):
- `vegas_scroll.switch_interval_ms` lowers the switch interval for a Vegas
run (1 ms is the obvious try), so the render thread waits at most that
long behind bytecode. It does nothing for a C call that keeps the GIL.
- `vegas_scroll.prefetch_gate` (`src/common/render_gate.py`) lets the
prefetch thread run Python only while the render thread is blocked in
`SwapOnVSync`, up to just before the refresh the swap returns on, and
parks it the rest of the time. That covers C calls too, since the gate is
checked before each one starts. It never parks the thread while it holds
a lock the render thread takes, and never for more than 50 ms. It needs
the rebuilt binding, which releases the GIL during the swap. On by
default.
## What this does not fix
- **The blit.** Copying a 512×64 frame into the matrix (`SetImage`) is ~6 ms at
8 PWM bits on a Pi 4, leaving ~4 ms of slack per refresh. That is the main
source of the single-refresh late frames. Holding frames for two refreshes
(≈50 px/s) doubles the budget. Cutting the blit itself is the native-presenter
step.
- **Live refreshes pushed from `update()`.** Some sports plugins call
`display()` and `update_display()` from inside `update()`, which runs on the
update worker and can push to the panel mid-Vegas. That is a separate
hazard. `offscreen()` gives a tool for it (run the update worker offscreen
while Vegas owns the panel), but it is out of scope here.
## Test plan
- **Unit, `DisplayManager`:** one thread inside `offscreen()` draws while
another reads `image`/`draw`/`matrix`/`width`/`height` and sees the real
canvas. Also: `update_display()` and `set_scrolling_state()` are inert inside;
`render_size()` narrows only the calling thread; nesting and exceptions
restore state.
- **Unit, adapter:** a stub display-capture plugin and a stub scroll-helper
plugin both return content with `offscreen_only=True`, and nothing is queued
for the render thread. The plugin lock is taken, and a timeout keeps the cached
segment.
- **Emulator integration:** a stub canvas-bound plugin whose `display()` sleeps
300 ms. The Vegas render loop never goes a frame without presenting (frame
timing recorder: zero freezes).
- **Unit, `SegmentStrip`:** the viewport assembled across segment boundaries
matches slicing one concatenated image, pixel for pixel. Append, trim,
replace-ahead and same-geometry patch each leave every other column
unchanged. A geometry-changing patch of a visible segment is refused.
- **Freshness:** a stub sports plugin whose score changes every second. The
score on screen is never older than `lead / speed` plus the plugin's fetch
interval. A visible card's digits change without the frame-timing recorder
seeing a late frame. An unchanged redraw is discarded.
- **Hardware:** an hdpi soak, A/B against the #628 build, alternating order.
Targets: no freezes, an empty 6+ bucket, the 3–5 bucket near zero, and the late
rate below 0.66%. Plus, for freshness: log each segment's age when it enters
the viewport, and compare the median and max before and after.
## Rollout
Three changes, each soaked on hdpi before the next:
1. **Offscreen rendering:** `offscreen()`, the adapter on the prefetch thread,
and the plugin lock. Removes the render-thread pauses.
2. **`SegmentStrip`:** Vegas's strip becomes a list of segments. Removes the
whole-strip copy on append. No visible behaviour change.
3. **Fresh content:** refresh at the gate, replace ahead, patch on screen,
with change detection and the rate limit.
`display.vegas_scroll.offscreen_prefetch` (default `true`) restores today's
deferred path when `false`, and `display.vegas_scroll.live_refresh` (default
`true`) turns off step 3. Keep both for one release, then delete the old paths.
## Open questions
1. Keep the kill switch, or ship without one?
2. Plugin lock timeout: skip the plugin and keep its cached segment (proposed),
or wait longer?
3. `refresh_min_interval`: 10 s proposed. It only limits chatty plugins;
live sports are redrawn once per 30 s poll regardless.
4. Multi-display sync: is there a two-Pi rig to test on? Operation replay is
proposed as part of the segment strip (step 2), with fresh content
disabled under sync until it has been verified on real hardware.
-154
View File
@@ -1,154 +0,0 @@
# Permissions
Who owns what on an installed system, which privileged commands the web
interface may run, and how to repair ownership when it goes wrong. The
installer, [`first_time_install.sh`](../first_time_install.sh), sets all of
this up; this page describes the result.
## Users and groups
| Account | Used by | Why |
|---|---|---|
| `root` | `ledmatrix.service` (the display) | The LED matrix library needs direct GPIO access |
| The installing user (e.g. `ledpi`) | `ledmatrix-web.service`, `ledmatrix-update-verify.service` | A web server should not run as root |
| `ledmatrix` group | shared files | Members: the installing user, `root`, and `daemon` if it exists. Created by [`setup_cache.sh`](../scripts/install/setup_cache.sh) and the installer |
The installer also adds the web user to `systemd-journal` and `adm` so the
**Logs** tab can read the journal. Group changes apply after the user logs
in again (services pick them up on restart).
## Files and directories
| Path | Owner | Mode | Notes |
|---|---|---|---|
| Project directory | web user | dirs `755`, files `644`, `*.sh` `755` | Set in the installer's "Normalize project file permissions" step |
| `config/` | web user | `2775` | |
| `config/config.json` | web user | `644` | Written by the web interface |
| `config/config_secrets.json` | web user : `ledmatrix` | `640` | Owned by the web user because the web interface writes it; root reads it regardless of mode |
| `plugin-repos/`, `plugins/` | web user | dirs `2775`, files `664` | The web interface installs and removes plugins |
| `assets/` | web user | dirs `755`, files `644` | Root writes downloaded logos regardless |
| `/var/cache/ledmatrix/` | `root:ledmatrix` | `2775` (setgid) | Shared cache: see below |
| Cache files | creator : `ledmatrix` | `660` | |
| `scripts/fix_perms/safe_plugin_rm.sh`, `safe_pip_install.sh` | `root:root` | `755` | Run as root through sudo, so the web user must not be able to edit them |
| `/etc/sudoers.d/ledmatrix_web`, `ledmatrix_wifi` | `root` | `440` | |
What keeps it that way at runtime:
- **Config files.** Saves go through
[`src/config_manager_atomic.py`](../src/config_manager_atomic.py), which
applies `get_config_file_mode()` (`640` for secrets, `644` otherwise) and,
when running as root, moves the file's group to the project directory's
group (`ensure_shared_group_ownership()` in
[`src/common/permission_utils.py`](../src/common/permission_utils.py)).
- **Cache files.** [`src/cache/disk_cache.py`](../src/cache/disk_cache.py)
sets every file it writes to `0660` and gives it the cache directory's
group, without relying on the setgid bit. So a file root writes stays
readable by the web user.
- **Plugin directories.** [`run.py`](../run.py) sets
`sys.dont_write_bytecode`, because root-owned `__pycache__` directories
inside a plugin stop the web user updating or removing it.
`ledmatrix-web.service` deliberately has no `CacheDirectory=`: systemd would
re-own `/var/cache/ledmatrix` to the web user and its primary group, and the
web interface could no longer read what the display writes (see the comment
in [`systemd/ledmatrix-web.service`](../systemd/ledmatrix-web.service)).
If `/var/cache/ledmatrix` is not usable, `CacheManager` falls back to
`~/.ledmatrix_cache`, `/opt/ledmatrix/cache` or a temp directory
([`src/cache_manager.py`](../src/cache_manager.py)). The two services then
may not share a cache, and the web UI shows stale or empty display status,
on-demand state and plugin health. Fix the directory rather than living
with the fallback.
## sudo rules
### `/etc/sudoers.d/ledmatrix_web`
Generated by `web_sudoers_rules()` in
[`scripts/install/lib_sudoers.sh`](../scripts/install/lib_sudoers.sh), the
only place these rules are defined. Installed by the installer and by
[`configure_web_sudo.sh`](../scripts/install/configure_web_sudo.sh), both of
which check them with `visudo -c` first. The web user may run, without a
password:
- `reboot`, `poweroff`
- `systemctl start|stop|restart|enable|disable|status ledmatrix.service`,
`systemctl is-active ledmatrix[.service]`
- `systemctl start|stop|restart ledmatrix-web.service`
- `bash <project>/scripts/fix_perms/safe_plugin_rm.sh *` — removes a
directory only if it resolves to a child of `plugin-repos/` or `plugins/`
- `bash <project>/scripts/fix_perms/safe_pip_install.sh *` — installs a
`requirements.txt` only if it is the project's own or one under
`plugin-repos/` or `plugins/`, so the root display service can import the
packages
- `journalctl -u ledmatrix.service *`, `-u ledmatrix *`, `-t ledmatrix *`,
tagged `NOEXEC`: journalctl opens a pager on a terminal, and a shell
escape from that pager would be a root shell
### `/etc/sudoers.d/ledmatrix_wifi`
Written by
[`scripts/install/configure_wifi_permissions.sh`](../scripts/install/configure_wifi_permissions.sh)
(run as the web user; the installer calls it). It refuses to grant a binary
that is not root-owned or is group/world-writable. The rules cover:
- `nmcli device wifi connect|disconnect *`, `nmcli device connect|disconnect *`,
`nmcli radio wifi on|off`
- `systemctl start|stop|restart hostapd`, `... dnsmasq`,
`systemctl restart NetworkManager`
- `sysctl -w net.ipv4.ip_forward=0|1`
- `nft add|delete table ip ledmatrix`
- `rfkill unblock wifi`
- `mkdir -p /etc/NetworkManager/dnsmasq-shared.d`
- `cp` of `/tmp/hostapd.conf` and `/tmp/dnsmasq.conf` to their fixed
destinations, and `rm -f /etc/dnsmasq.d/ledmatrix-captive.conf`
- `cp /tmp/ledmatrix-nm-dnsmasq.conf` to
`/etc/NetworkManager/dnsmasq-shared.d/ledmatrix-captive.conf`, and
`rm -f` of that file
**`iptables` is deliberately not granted.** The captive portal's rules are
built from the interface name and port, so a rule covering them would need a
trailing wildcard, and `iptables --modprobe=<path>` runs `<path>` as root: a
wildcard grant is a root shell for the web user. Doing it safely needs a
wrapper script that builds the rules itself, like `safe_plugin_rm.sh`. On a
stock Raspberry Pi OS image the default user's blanket `NOPASSWD` rule
(`/etc/sudoers.d/010_pi-nopasswd`) hides this gap.
### polkit
The same script installs `/etc/polkit-1/rules.d/10-ledmatrix-wifi.rules`,
which lets the web user perform any `org.freedesktop.NetworkManager.*`
action without authentication.
## Repair scripts
In [`scripts/fix_perms/`](../scripts/fix_perms/). Run them from the project
directory.
| Script | Run as | What it does | Notes |
|---|---|---|---|
| `fix_plugin_permissions.sh` | `sudo` | `plugins/` and `plugin-repos/` to `root:<user>`, dirs `2775`, files `664`; makes a `700` home directory `755` so root can traverse it | Safe. Group-writable, so the web user keeps write access |
| `fix_assets_permissions.sh` | `sudo` | `assets/` to `<user>:<group>`, mode `777` recursively | Works, but looser than the installer's `755`/`644` |
| `fix_cache_permissions.sh` | `sudo` | Runs [`setup_cache.sh`](../scripts/install/setup_cache.sh) for `/var/cache/ledmatrix` (`root:ledmatrix`, `2775`, files `660`), then makes `~/.ledmatrix_cache` (the fallback cache) `<user>:<group>` mode `777` | Safe. The `~/.ledmatrix_cache` mode is still `777` |
| `fix_web_permissions.sh` | the web user, **without** `sudo` | Resets project file ownership for the web user (it calls `sudo` itself), then makes `safe_plugin_rm.sh` and `safe_pip_install.sh` `root:root` `755` again and restores `config_secrets.json` to its owner, group `ledmatrix`, mode `640` | Refuses to run as root. It does not write sudoers rules |
| `safe_plugin_rm.sh`, `safe_pip_install.sh` | — | Called by the web interface through sudo | Not for manual use |
To reinstall the sudoers rules, run
`./scripts/install/configure_web_sudo.sh` (web rules) or
`./scripts/install/configure_wifi_permissions.sh` (WiFi rules and polkit) as
the web user, not with `sudo`.
After any of these, restart both services:
```bash
sudo systemctl restart ledmatrix.service ledmatrix-web.service
```
## Checking
```bash
ls -ld /var/cache/ledmatrix # drwxrwsr-x root ledmatrix
stat -c '%U:%G %a %n' config/config.json config/config_secrets.json
id # web user should list ledmatrix
sudo -l # lists the NOPASSWD rules
```
+81 -72
View File
@@ -9,7 +9,6 @@ Complete API reference for plugin developers. This document describes all method
## Table of Contents
- [Manifest Required Fields](#manifest-required-fields)
- [BasePlugin](#baseplugin)
- [Display Manager](#display-manager)
- [Cache Manager](#cache-manager)
@@ -18,50 +17,6 @@ Complete API reference for plugin developers. This document describes all method
---
## Manifest Required Fields
Three parts of core check `manifest.json`, each for a different set of
fields:
| Check | Fields | What happens when one is missing |
|---|---|---|
| JSON schema, [`schema/manifest_schema.json`](../schema/manifest_schema.json) | `id`, `name`, `version`, `author`, `entry_point`, `class_name`, `compatible_versions` | Install from URL logs a warning (`PluginStoreManager._validate_manifest_schema()`); nothing is refused |
| Plugin Store install, [`src/plugin_system/store_install.py`](../src/plugin_system/store_install.py) | `id`, `name`, `class_name`, `display_modes` | Install is refused. A registry install first tries to detect a missing `class_name` from the entry-point file |
| Plugin loader, [`src/plugin_system/plugin_loader.py`](../src/plugin_system/plugin_loader.py) | `class_name` | The plugin fails to load |
Defaults and other uses:
- `entry_point` defaults to `manager.py`; the store writes the default back
into the manifest on install.
- `compatible_versions` (a list of semver ranges such as `">=2.0.0"`) is how
the store decides whether a plugin can run on this core. An install is
refused only when the field excludes the running version
(`compatibility.check()` in
[`src/plugin_system/compatibility.py`](../src/plugin_system/compatibility.py)).
- `version` is compared with the registry's `latest_version` to decide
whether an update is available.
- If `display_modes` is empty at load time, the display controller uses the
plugin id as the only mode.
**Set all eight:** `id`, `name`, `version`, `author`, `entry_point`,
`class_name`, `display_modes`, `compatible_versions`. That satisfies every
check. The schema lists the optional fields.
```json
{
"id": "my-plugin",
"name": "My Plugin",
"version": "1.0.0",
"author": "YourName",
"entry_point": "manager.py",
"class_name": "MyPlugin",
"display_modes": ["my-plugin"],
"compatible_versions": [">=2.0.0"]
}
```
---
## BasePlugin
All plugins must inherit from `BasePlugin` and implement the required methods. The base class provides access to managers and common functionality.
@@ -475,19 +430,86 @@ self.display_manager.image.paste(icon, (5, 5), icon)
self.display_manager.update_display()
```
This is the canonical way to render arbitrary images.
This is the same pattern the bundled scoreboard base classes
(`src/base_classes/baseball.py`, `basketball.py`, `football.py`,
`hockey.py`) use, so it's the canonical way to render arbitrary images.
### Weather Icons (deprecated)
### Weather Icons
> Deprecated, removed in 3.7.0 — draw your own icons (the weather plugin
> ships `WeatherIcons`). See [Deprecated APIs](#deprecated-apis).
#### `draw_weather_icon(condition: str, x: int, y: int, size: int = 16) -> None`
- `draw_weather_icon(condition, x, y, size=16)` — icon for a condition
string such as `"clear"`, `"clouds"`, `"rain"`, `"snow"`, `"storm"`
- `draw_sun(x, y, size=16)`, `draw_cloud(x, y, size=16, color=(200, 200, 200))`,
`draw_rain(x, y, size=16)`, `draw_snow(x, y, size=16)`
- `draw_text_with_icons(text, icons=None, x=None, y=None, color=(255, 255, 255))`
— text plus a list of `(icon_type, x, y)` icons; calls `update_display()`
Draw a weather icon based on the condition string.
**Parameters**:
- `condition` (str): Weather condition (e.g., "clear", "cloudy", "rain", "snow", "storm")
- `x` (int): X position
- `y` (int): Y position
- `size` (int): Icon size in pixels (default: 16)
**Supported Conditions**:
- `"clear"`, `"sunny"` → Sun icon
- `"clouds"`, `"cloudy"`, `"partly cloudy"` → Cloud icon
- `"rain"`, `"drizzle"`, `"shower"` → Rain icon
- `"snow"`, `"sleet"`, `"hail"` → Snow icon
- `"thunderstorm"`, `"storm"` → Storm icon
**Example**:
```python
self.display_manager.draw_weather_icon("rain", x=10, y=10, size=16)
```
#### `draw_sun(x: int, y: int, size: int = 16) -> None`
Draw a sun icon with rays.
**Parameters**:
- `x` (int): X position
- `y` (int): Y position
- `size` (int): Icon size (default: 16)
#### `draw_cloud(x: int, y: int, size: int = 16, color: tuple = (200, 200, 200)) -> None`
Draw a cloud icon.
**Parameters**:
- `x` (int): X position
- `y` (int): Y position
- `size` (int): Icon size (default: 16)
- `color` (tuple): RGB color (default: light gray)
#### `draw_rain(x: int, y: int, size: int = 16) -> None`
Draw rain icon with cloud and droplets.
#### `draw_snow(x: int, y: int, size: int = 16) -> None`
Draw snow icon with cloud and snowflakes.
#### `draw_text_with_icons(text: str, icons: List[tuple] = None, x: int = None, y: int = None, color: tuple = (255, 255, 255)) -> None`
Draw text with weather icons at specified positions.
**Parameters**:
- `text` (str): Text to display
- `icons` (List[tuple], optional): List of (icon_type, x, y) tuples
- `x` (int, optional): X position for text
- `y` (int, optional): Y position for text
- `color` (tuple): Text color
**Note**: Automatically calls `update_display()` after drawing.
**Example**:
```python
icons = [
("sun", 5, 5),
("cloud", 100, 5)
]
self.display_manager.draw_text_with_icons(
"Weather: Sunny, Cloudy",
icons=icons,
x=10, y=20
)
```
### Scrolling State Management
@@ -581,8 +603,6 @@ Process any deferred updates if not currently scrolling. Called automatically by
#### `get_scrolling_stats() -> dict`
> Deprecated, removed in 3.7.0. See [Deprecated APIs](#deprecated-apis).
Get current scrolling statistics for debugging.
**Returns**: Dictionary with scrolling state information
@@ -724,8 +744,6 @@ data = self.cache_manager.get_with_auto_strategy("nhl_live_scores")
#### `get_background_cached_data(key: str, sport_key: Optional[str] = None) -> Optional[Dict[str, Any]]`
> Deprecated, removed in 3.7.0 — use `get()`. See [Deprecated APIs](#deprecated-apis).
Get background service cached data with sport-specific intervals.
**Parameters**:
@@ -763,8 +781,6 @@ max_age = strategy['max_age'] # Get configured max age
#### `get_sport_live_interval(sport_key: str) -> int`
> Deprecated, removed in 3.7.0. See [Deprecated APIs](#deprecated-apis).
Get the live_update_interval for a specific sport from config.
**Parameters**:
@@ -789,8 +805,6 @@ Extract data type from cache key to determine appropriate cache strategy.
#### `get_sport_key_from_cache_key(key: str) -> Optional[str]`
> Deprecated, removed in 3.7.0. See [Deprecated APIs](#deprecated-apis).
Extract sport key from cache key for sport-specific strategies.
**Parameters**:
@@ -835,12 +849,10 @@ for file_info in files:
self.logger.info(f"Cache: {file_info['key']}, Age: {file_info['age_display']}")
```
### Metrics Methods (deprecated)
### Metrics Methods
#### `get_cache_metrics() -> Dict[str, Any]`
> Deprecated, removed in 3.7.0. See [Deprecated APIs](#deprecated-apis).
Get cache performance metrics.
**Returns**: Dictionary with cache statistics (`total_requests`, `cache_hit_rate`, `background_hit_rate`, `api_calls_saved`, `average_fetch_time`, etc.)
@@ -853,8 +865,6 @@ self.logger.info(f"Cache hit rate: {metrics['cache_hit_rate']:.2%}")
#### `get_memory_cache_stats() -> Dict[str, Any]`
> Deprecated, removed in 3.7.0. See [Deprecated APIs](#deprecated-apis).
Get memory cache statistics.
**Returns**: Dictionary with memory cache stats (size, max_size, etc.)
@@ -899,8 +909,6 @@ for plugin_id, plugin in all_plugins.items():
#### `get_enabled_plugins() -> List[str]`
> Deprecated, removed in 3.7.0 — check `enabled` on the instances in `plugin_manager.plugins`. See [Deprecated APIs](#deprecated-apis).
Get list of enabled plugin IDs.
**Returns**: List of plugin identifier strings
@@ -979,8 +987,9 @@ def update(self):
**Example - Checking if another plugin is enabled**:
```python
weather = self.plugin_manager.plugins.get("weather")
if weather is not None and weather.enabled:
enabled_plugins = self.plugin_manager.get_enabled_plugins()
if "weather" in enabled_plugins:
# Weather plugin is enabled
pass
```
+3 -3
View File
@@ -12,9 +12,9 @@
> in `web_interface/app.py`.
> - The default plugin location is `plugin-repos/` (configurable via
> `plugin_system.plugins_directory`), not `./plugins/`.
> - Example imports use `src/plugin_system/base_classes/*_plugin.py`,
> which do not exist. The old `src/base_classes/` package has been
> removed; shared sports code lives in `src/common/`.
> - Example imports use `src/plugin_system/base_classes/*_plugin.py`;
> the shipped base classes live in `src/base_classes/` (e.g.
> `src.base_classes.sports.SportsCore`, `src.base_classes.hockey.Hockey`).
> - The "Migration Strategy" and "Implementation Roadmap" sections
> describe work that has now shipped.
>
+1 -9
View File
@@ -124,14 +124,6 @@ Plugins are configured by adding their plugin ID as a top-level key in the confi
}
```
How often the core calls a plugin's `update()`: the plugin's
`get_update_interval()` if it returns a number, else `update_interval` in the
plugin's `manifest.json`, else `update_interval` in its `config.json` section
as above, else 60 seconds. A config `update_interval` therefore only sets the
scheduler's cadence for a plugin whose manifest does not; plugins that expose
it in their config schema typically also honour it themselves inside
`update()`. See [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md#get_update_interval---optionalfloat).
### Plugin Display Durations
Add plugin display modes to the `display_durations` section:
@@ -202,7 +194,7 @@ plugin-repos/
```
The Plugin Store refuses a manifest that lacks any of `id`, `name`,
`class_name` or `display_modes` (`store_install.py`); the loader itself
`class_name` or `display_modes` (`store_manager.py`); the loader itself
needs `class_name`. `version` is not required, but the store compares it
with the registry's `latest_version` to offer updates, so set it.
`entry_point` defaults to `manager.py` if omitted. The config schema is not
+1 -1
View File
@@ -203,7 +203,7 @@ Forms are rendered on the server, not generated in the browser:
from the schema (widgets named by `x-widget` are rendered by the scripts in
`web_interface/static/v3/js/widgets/`)
4. **Save Configuration** posts the form to `/api/v3/plugins/config`
(`web_interface/blueprints/api_v3/plugin_config.py`), which validates it against
(`web_interface/blueprints/api_v3/plugins.py`), which validates it against
the schema, writes `config.json` (secret fields go to
`config_secrets.json`) and shows a notification
+6 -6
View File
@@ -32,7 +32,7 @@
│ • masks x-secret fields │
│ • renders partials/plugin_config.html (render_field macros) │
│ │
│ api_v3 blueprint (blueprints/api_v3/plugin_config.py) │
│ api_v3 blueprint (blueprints/api_v3/plugins.py) │
│ save_plugin_config() POST /api/v3/plugins/config │
│ get_plugin_config() GET /api/v3/plugins/config │
│ get_plugin_schema() GET /api/v3/plugins/schema │
@@ -91,7 +91,7 @@ validatePluginConfigForm() (client-side checks)
POST /api/v3/plugins/config?plugin_id=<id> (form data, all fields of the form)
│
▼
save_plugin_config() (api_v3/plugin_config.py)
save_plugin_config() (api_v3/plugins.py)
├─→ Start from the stored config.json[<id>]
├─→ Apply form fields: dotted names → nested keys, "[]" checkbox
│ groups → lists, values coerced to the schema's types
@@ -155,9 +155,9 @@ deep-merged back into the plugin's config at load time
### Custom input widgets
Set `"x-widget": "<name>"` on a property. Core widgets are in
`web_interface/static/v3/js/widgets/`; a plugin can ship its own widget
script, served from `/static/plugin-widgets/<plugin_id>/<name>.js`. See the
[widget guide](../web_interface/static/v3/js/widgets/README.md).
`web_interface/static/v3/js/widgets/` (see its README); a plugin can ship its
own widget script, served from `/static/plugin-widgets/<plugin_id>/<name>.js`.
See [widget-guide.md](widget-guide.md).
### Custom actions
@@ -175,7 +175,7 @@ Implement `on_config_change(new_config)` in the plugin (see
|---------|------|
| Tab partial loader | `web_interface/blueprints/pages_v3.py` (`_load_plugin_config_partial`) |
| Form template and field macros | `web_interface/templates/v3/partials/plugin_config.html` |
| Save / get / schema / reset handlers | `web_interface/blueprints/api_v3/plugin_config.py` |
| Save / get / schema / reset handlers | `web_interface/blueprints/api_v3/plugins.py` |
| Schema loading, defaults, validation | `src/plugin_system/schema_manager.py` |
| Secret masking and splitting | `src/web_interface/secret_helpers.py` |
| Widgets | `web_interface/static/v3/js/widgets/` |
+10 -5
View File
@@ -25,7 +25,16 @@ is `CORE_PLUGIN_PROPERTIES` in `src/plugin_system/schema_manager.py`):
- Description: Enable live priority takeover when plugin has live content
- Used by DisplayController for priority scheduling
4. **`vegas_width_pct`**, **`vegas_overflow`**, **`vegas_max_width_screens`**
4. **`skin`** (string, object or null; no default)
- Description: Visual skin id, or a per-mode mapping like `{"live": "my-skin"}`
- Not an enum, so a stored value keeps validating after the skin is
uninstalled. Skins do not render with the current scoreboard plugins;
the key is kept so stored values keep loading and saving
5. **`skin_options`** (object; no default)
- Description: Options passed through to the selected skin
6. **`vegas_width_pct`**, **`vegas_overflow`**, **`vegas_max_width_screens`**
(untyped; no default)
- Description: Vegas mode tuning for this plugin — card width as a
percentage of the panel, `"rotate"` or `"truncate"` on overflow, and the
@@ -33,10 +42,6 @@ is `CORE_PLUGIN_PROPERTIES` in `src/plugin_system/schema_manager.py`):
- Read by `src/vegas_mode/plugin_adapter.py` and `BasePlugin`, which
validate the values themselves and ignore a bad one with a log line
`skin` and `skin_options` were core properties until the skin system was
removed. A plugin config saved with them still loads and saves; the keys are
dropped on the next save (see `RETIRED_PLUGIN_KEYS` in `schema_manager.py`).
## How Core Properties Work
### Schema Validation
+7 -5
View File
@@ -5,9 +5,11 @@
A plugin can name an icon for its tab in the web interface's second nav row
(next to **Plugin Manager**) with the `icon` field in `manifest.json`.
`GET /api/v3/plugins/installed` passes the manifest's `icon` through (a
non-string value comes back as `null`), and a plugin without one gets the
default puzzle piece.
> **Status:** the tab code honors `icon`, but `GET /api/v3/plugins/installed`
> (`web_interface/blueprints/api_v3/plugins.py`) does not currently include
> the manifest's `icon` in its response, so every tab shows the default
> puzzle piece. Setting `icon` is harmless and will take effect once the API
> passes it through again.
## Font Awesome classes only
@@ -53,8 +55,8 @@ With no `icon` (or an empty one) the tab shows `fas fa-puzzle-piece`.
or misspelled class renders as a blank space.
2. Include the style prefix (`fas`, `far` or `fab`) as well as the icon
class.
3. The manifest is re-read on each plugin list load; reload the page after
editing `icon`.
3. See the status note above: the icon is currently not passed through by
the API.
## Related Documentation
+3 -4
View File
@@ -25,7 +25,7 @@ which runs as root.** Anything installed only into another user's
The web interface is not root, so it installs through a narrow sudo helper:
1. `PluginStoreManager._install_dependencies()`
(`src/plugin_system/store_install.py`) calls
(`src/plugin_system/store_manager.py`) calls
`install_requirements_file()` (`src/common/permission_utils.py`).
2. That runs `sudo -n bash scripts/fix_perms/safe_pip_install.sh <plugin>/requirements.txt`.
The helper checks the path is the project's own `requirements.txt` or a
@@ -154,9 +154,8 @@ For more, see the [Plugin Dependency Troubleshooting Guide](PLUGIN_DEPENDENCY_TR
## Files to Reference
- Service units: `systemd/ledmatrix.service`, `systemd/ledmatrix-web.service`
- Store installs: `src/plugin_system/store_install.py` (`_install_dependencies`)
- Store installs: `src/plugin_system/store_manager.py` (`_install_dependencies`)
- Root install helper: `src/common/permission_utils.py` (`install_requirements_file`), `scripts/fix_perms/safe_pip_install.sh`
- Load-time installs: `src/plugin_system/plugin_loader.py` (`install_dependencies`)
- Sudo rules: `scripts/install/lib_sudoers.sh` (written by `first_time_install.sh`
and `scripts/install/configure_web_sudo.sh`)
- Sudo rules: `scripts/install/configure_web_sudo.sh`
- Manual installer: `scripts/install_plugin_dependencies.sh`
+11 -10
View File
@@ -12,6 +12,12 @@ This guide explains how to set up a development workflow for plugins that are ma
> scale. Existing plugins keep their classic rendering unless they adopt
> those APIs; nothing migrates automatically.
> **Want a different look for an existing sports scoreboard?** Skins are
> meant for that, but they are **not supported yet**: the current scoreboard
> plugins don't render them (see [SKIN_SYSTEM.md](SKIN_SYSTEM.md#status-not-supported-yet)).
> For now, change the look through the plugin's own display settings or its
> code.
## Overview
When developing plugins in separate repositories, you need a way to:
@@ -520,22 +526,19 @@ When developing plugins, you'll need to use the APIs provided by the LEDMatrix s
`display_manager.image` (a PIL Image) and call `update_display()`;
there is no `draw_image()` helper method.
- `draw_weather_icon()`, `draw_sun()`, `draw_cloud()` - Weather icons
(deprecated, removed in 3.7.0 — draw your own icons)
- `get_text_width()`, `get_font_height()` - Text utilities
- `set_scrolling_state()`, `defer_update()` - Scrolling state management
**Cache Manager** (`self.cache_manager`):
- `get()`, `set()`, `delete()` - Basic caching
- `get_cached_data_with_strategy()` - Advanced caching with strategies
- `get_background_cached_data()` - deprecated, removed in 3.7.0 — use `get()`
- `get_background_cached_data()` - Background service caching
**Plugin Manager** (`self.plugin_manager`):
- `get_plugin()`, `get_all_plugins()` - Access other plugins
- `get_plugin_info()` - Get plugin information
See [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md) for complete
documentation, and its [Deprecated APIs](PLUGIN_API_REFERENCE.md#deprecated-apis)
table for everything removed in 3.7.0.
See [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md) for complete documentation.
## 3rd Party Plugin Development
@@ -580,14 +583,12 @@ Your plugin must:
pass
```
2. **Include manifest.json** with the required fields listed in
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md#manifest-required-fields):
2. **Include manifest.json** with required fields:
```json
{
"id": "my-plugin",
"name": "My Plugin",
"version": "1.0.0",
"author": "YourName",
"class_name": "MyPlugin",
"entry_point": "manager.py",
"display_modes": ["my_plugin"],
@@ -645,7 +646,7 @@ To have your plugin added to the official plugin store:
3. **Contact maintainers** (own-repository plugins):
- Open a GitHub issue in the [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) repository
- Or reach out on Discord: https://discord.gg/RdrC37rEag
- Or reach out on Discord: https://discord.gg/uW36dVAtcT
- Include: Repository URL, plugin description, why it's useful
4. **Review process**:
@@ -663,7 +664,7 @@ For your plugin to work well in the plugin store:
with the registry's `latest_version`; releases and tags are not read
- **README.md**: Clear installation and configuration instructions
- **config_schema.json**: Recommended for web UI configuration
- **manifest.json**: Required, with the [required fields](PLUGIN_API_REFERENCE.md#manifest-required-fields)
- **manifest.json**: Required with all required fields
- **requirements.txt**: If your plugin has Python dependencies
### Distribution Options
+2 -16
View File
@@ -146,10 +146,7 @@ def display(self, force_clear: bool = False) -> bool:
## Error Aggregation
LEDMatrix automatically tracks plugin errors: every exception or timeout from
a plugin's `update()` or `display()` is recorded by the display service,
which runs the plugins. See them in the web interface under **Logs → Plugin
errors**, or through the API:
LEDMatrix automatically tracks plugin errors. Access error data via the API:
```bash
# Get error summary
@@ -158,21 +155,10 @@ curl http://localhost:5000/api/v3/errors/summary
# Get plugin-specific health
curl http://localhost:5000/api/v3/errors/plugin/my-plugin
# Clear errors older than 24 hours (the default), or all of them
# Clear old errors
curl -X POST http://localhost:5000/api/v3/errors/clear
curl -X POST -H 'Content-Type: application/json' -d '{"all": true}' \
http://localhost:5000/api/v3/errors/clear
```
The web interface is a separate process, so it reads a snapshot the display
service writes to the shared cache directory (`plugin_error_snapshot`): at most
every 10 seconds, and only when something changed. Expect the numbers to lag
by up to about 15 seconds, and to start from zero when the display service
restarts. `snapshot_available` is `false` until the display service has
reported. A clear is a request the display service applies within about 5
seconds; the API hides the cleared errors immediately. Details and response
shapes: [REST API reference](REST_API_REFERENCE.md#error-tracking).
### Error Patterns
When the same error occurs repeatedly (5+ times in 60 minutes), it's detected as a pattern and logged as a warning. This helps identify systemic issues.
+1 -4
View File
@@ -45,8 +45,7 @@ LEDMatrix/
### 1. Minimal Plugin Structure
**manifest.json** (the required fields are explained in
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md#manifest-required-fields)):
**manifest.json**:
```json
{
"id": "my-plugin",
@@ -55,8 +54,6 @@ LEDMatrix/
"author": "YourName",
"entry_point": "manager.py",
"class_name": "MyPlugin",
"display_modes": ["my-plugin"],
"compatible_versions": [">=2.0.0"],
"category": "custom"
}
```
+3 -3
View File
@@ -67,9 +67,9 @@ Don't edit `latest_version` or `last_updated` by hand for monorepo plugins:
## Adding or changing an official plugin
1. Add or edit `plugins/<your-plugin-id>/` in the monorepo, with the
manifest fields listed in
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md#manifest-required-fields).
1. Add or edit `plugins/<your-plugin-id>/` in the monorepo. The store refuses
a manifest without `id`, `name`, `class_name` and `display_modes`; also
set `version`.
2. Bump `version` in the plugin's `manifest.json` for every change, or users
won't be offered the update.
3. Run `python update_registry.py` in ledmatrix-plugins and commit the
+5 -9
View File
@@ -11,7 +11,7 @@ the one-shot installer. The pages here go deeper.
2. [WEB_INTERFACE_GUIDE.md](WEB_INTERFACE_GUIDE.md) — using the web UI
3. [PLUGIN_STORE_GUIDE.md](PLUGIN_STORE_GUIDE.md) — installing and managing plugins
4. [WIFI_NETWORK_SETUP.md](WIFI_NETWORK_SETUP.md) — WiFi and AP-mode setup
5. [TROUBLESHOOTING.md](TROUBLESHOOTING.md) — common issues and fixes ([PERMISSIONS.md](PERMISSIONS.md) for "Permission denied")
5. [TROUBLESHOOTING.md](TROUBLESHOOTING.md) — common issues and fixes
6. [SSH_UNAVAILABLE_AFTER_INSTALL.md](SSH_UNAVAILABLE_AFTER_INSTALL.md) — recovering SSH after install
7. [CONFIG_DEBUGGING.md](CONFIG_DEBUGGING.md) — diagnosing config problems
8. [LOW_MEMORY_BOARDS.md](LOW_MEMORY_BOARDS.md) — Pi Zero 2 W / 3B+ / 1GB Pi 4 memory limits
@@ -37,7 +37,7 @@ Going deeper:
- [PLUGIN_CUSTOM_ICONS.md](PLUGIN_CUSTOM_ICONS.md)
- [PLUGIN_REGISTRY_SETUP_GUIDE.md](PLUGIN_REGISTRY_SETUP_GUIDE.md) (+ [registry template](plugin_registry_template.json))
- [STARLARK_APPS_GUIDE.md](STARLARK_APPS_GUIDE.md) — Starlark-based mini-apps
- [Widget guide](../web_interface/static/v3/js/widgets/README.md) — built-in `x-widget`s and custom widgets
- [widget-guide.md](widget-guide.md) — widget development
- [ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md) — render legibly on any panel size (opt-in font/layout scaling)
- [plugin-safety-harness.md](plugin-safety-harness.md) — test a plugin across every screen and matrix size
@@ -56,27 +56,23 @@ Going deeper:
- [ADVANCED_FEATURES.md](ADVANCED_FEATURES.md) — Vegas scroll, on-demand display,
cache management, background services, permissions
- [FONT_MANAGER.md](FONT_MANAGER.md) — font system
- [SCROLL_PERFORMANCE.md](SCROLL_PERFORMANCE.md) — how scrolling is paced, and how to make a plugin's marquee smooth
- [OFFSCREEN_RENDERING.md](OFFSCREEN_RENDERING.md) — rendering plugin content off the render thread
- [PERMISSIONS.md](PERMISSIONS.md) — file ownership, sudo rules, repair scripts
- [MQTT bridge](../integrations/mqtt_bridge/README.md) — control the display from Home Assistant over MQTT
- [SKIN_SYSTEM.md](SKIN_SYSTEM.md) — skin architecture for sports scoreboards (not supported yet: current scoreboards don't render skins)
- [CREATING_SKINS.md](CREATING_SKINS.md) — writing and validating a skin (same caveat)
## Reference
- [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md) — every key in config.json and config_secrets.json
- [REST_API_REFERENCE.md](REST_API_REFERENCE.md) — all web-interface HTTP endpoints
- [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md) — Python APIs available to plugins
- [src/common/README.md](../src/common/README.md) — shared helper modules plugins can import
- [DEVELOPER_QUICK_REFERENCE.md](DEVELOPER_QUICK_REFERENCE.md) — common dev tasks
## Contributing to LEDMatrix itself
- [ARCHITECTURE.md](ARCHITECTURE.md) — processes, display loop, plugin system, web UI; where to start reading
- [DEVELOPMENT.md](DEVELOPMENT.md) — environment setup
- [HOW_TO_RUN_TESTS.md](HOW_TO_RUN_TESTS.md) — running the test suite
- [MULTI_ROOT_WORKSPACE_SETUP.md](MULTI_ROOT_WORKSPACE_SETUP.md) — multi-repo workspace
- [MIGRATION_GUIDE.md](MIGRATION_GUIDE.md) — breaking changes between releases
- [SPORTS_UNIFICATION.md](SPORTS_UNIFICATION.md) — how shared sports scoreboard code moves into `src/common`
- [SPORTS_UNIFICATION.md](SPORTS_UNIFICATION.md) — how the sports scoreboard base classes are organized
## Audits
+23 -97
View File
@@ -37,17 +37,17 @@ the entry below says so.
- [Integrations](#integrations)
- [Plugin-specific endpoints](#plugin-specific-endpoints)
- [Starlark Apps](#starlark-apps)
- [Skins](#skins)
> The API blueprint is the `api_v3` package in
> `web_interface/blueprints/api_v3/` (one module per area: `config.py`,
> `display.py`, `system.py`, `backup.py`, `fonts.py`, `misc.py`, `wifi.py`,
> `starlark.py`, and `plugins.py` plus the `plugin_*.py` modules for the
> plugin routes). `web_interface/app.py` registers it
> `display.py`, `plugins.py`, `system.py`, `backup.py`, `fonts.py`,
> `misc.py`, `wifi.py`, `starlark.py`). `web_interface/app.py` registers it
> at `/api/v3` (`app.register_blueprint(api_v3, url_prefix='/api/v3')`).
> The three SSE endpoints (`/api/v3/stream/*`) are defined directly on the
> Flask app in `app.py` (`stream_stats`, `stream_display`, `stream_logs`).
> `test/fixtures/api_v3_url_map.json` is the canonical list of blueprint
> routes; a test fails if the code and that fixture differ.
> routes (116 URL rules); a test fails if the code and that fixture differ.
---
@@ -870,13 +870,7 @@ Get a plugin's resource limits. `data` is `null` when none are configured.
**POST** `/api/v3/plugins/limits/<plugin_id>`
Set a plugin's resource limits. The body replaces all four limits: a key you
omit is stored as no limit (`warning_threshold` defaults to `0.8`). Each value
must be a non-negative number or `null`; anything else is a 400.
The limits are stored in the shared cache. A display service that has already
read limits for the plugin keeps using those until it restarts; likewise the
health and metrics reset routes clear the stored record and the web process's
copy, not the display service's in-memory state.
omit is stored as no limit (`warning_threshold` defaults to `0.8`).
**Request Body**:
```json
@@ -1837,73 +1831,25 @@ The last 100 journal lines for `ledmatrix.service` and
## Error tracking
Plugin errors are recorded by the display service (`ledmatrix.service`),
which runs the plugins. It publishes a snapshot to the shared cache directory
(`plugin_error_snapshot`) at most every 10 seconds, and only when something
changed, so these endpoints lag the display by up to about 15 seconds. The
counts cover the display service's current run: they start at zero when it
restarts. Error messages and stack traces have credentials redacted, and
messages, traces and context values are truncated in the snapshot.
Every response below adds three fields to the shape it always had:
| Field | Meaning |
|---|---|
| `snapshot_available` | `false` until the display service has reported (for example, it is not running). Counts are then zero. |
| `generated_at` | When the display service produced the snapshot (ISO, the Pi's local time), or `null`. |
| `clear_pending` | A clear has been requested and the display service has not applied it yet. |
### Get Error Summary
**GET** `/api/v3/errors/summary`
Aggregated counts, detected patterns and recent errors (the last 20).
```json
{
"status": "success",
"data": {
"session_start": "2026-09-23T09:40:02.118000",
"total_errors": 13,
"error_rate_per_hour": 41.2,
"error_counts_by_type": {"ConnectionError": 12, "ValueError": 1},
"plugin_error_counts": {"weather": {"ConnectionError": 12}, "stocks": {"ValueError": 1}},
"active_patterns": {
"ConnectionError": {
"error_type": "ConnectionError", "count": 12,
"first_seen": "2026-09-23T09:41:10.500000", "last_seen": "2026-09-23T09:58:36.020000",
"affected_plugins": ["weather"], "sample_messages": ["Read timed out."],
"severity": "error"
}
},
"recent_errors": [
{"error_type": "ValueError", "message": "could not parse price",
"timestamp": "2026-09-23T09:58:36.100000", "context": {},
"plugin_id": "stocks", "operation": "update", "stack_trace": "Traceback ..."}
],
"generated_at": "2026-09-23T09:58:40.000000",
"snapshot_available": true,
"clear_pending": false
},
"message": "Error summary retrieved"
}
```
Aggregated counts, detected patterns and recent errors across plugins and
core components.
### Get Plugin Errors
**GET** `/api/v3/errors/plugin/<plugin_id>`
Error health and statistics for one plugin: `plugin_id`, `status`
(`healthy`, `degraded` or `unhealthy`), `total_errors`, `error_types`,
`recent_error_count`, `last_error` (a `recent_errors` entry or `null`), plus
the three fields above. A plugin with no recorded errors is `healthy`.
Error health and statistics for one plugin.
### Clear Errors
**POST** `/api/v3/errors/clear`
Clear error records older than `max_age_hours` (default 24, 1-8760), or every
error with `"all": true` (`max_age_hours` is then ignored).
Clear error records older than `max_age_hours` (default 24, 1-8760).
Returns `data.cleared_count`.
```json
{
@@ -1911,32 +1857,6 @@ error with `"all": true` (`max_age_hours` is then ignored).
}
```
The clear is asynchronous. The web interface records a request
(`plugin_error_clear_request` in the shared cache), and the display service
applies it within about 5 seconds, rebuilding its counts from the errors it
keeps and republishing. Reads hide the cleared errors from the moment the
request is recorded. Until the display service applies an age-based clear,
`recent_errors` and `active_patterns` are already filtered but the counts
are the old ones, and `clear_pending` is `true`.
```json
{
"status": "success",
"data": {
"cleared_count": 13,
"clear_requested": true,
"request_id": "5f0c1e...",
"cutoff": "2026-09-23T09:59:02.310000"
},
"message": "Clear of all errors requested; the display service applies it within about 5 seconds"
}
```
`cleared_count` is how many of the reported errors the clear hides. It is
`null` when that cannot be known before the display service applies it (an
age-based clear over more errors than the report lists). A request that
could not be written to the shared cache answers `500`.
---
## Health and Status
@@ -2106,6 +2026,17 @@ runs.
---
## Skins
**GET** `/api/v3/skins`
Installed scoreboard skins (optional `?plugin_id=` filter). Skins are not
supported by the current scoreboard plugins, so the response carries
`data.supported: false` and a `data.message`; clients must not offer these
as selectable. See [SKIN_SYSTEM.md](SKIN_SYSTEM.md).
---
## Error Responses
Errors use one of two shapes. Most endpoints answer:
@@ -2118,18 +2049,13 @@ Errors use one of two shapes. Most endpoints answer:
}
```
An exception no route anticipated gets this shape too, with a 500, the
message `An error occurred; see logs for details`, and `details` naming the
exception type and text (credentials redacted). The api_v3 blueprint's
error handler produces it, so it is the same for every `/api/v3` route.
Endpoints built on the structured error helper add a code, and usually
suggested fixes (the web UI's error dialog lists them):
Endpoints built on the structured error helper add a code and category:
```json
{
"status": "error",
"error_code": "CONFIG_SAVE_FAILED",
"error_category": "configuration",
"message": "Error description",
"details": "optional",
"context": { },
+1 -270
View File
@@ -207,10 +207,7 @@ Fixed by rebuilding the binding: `scripts/build_rgbmatrix_nogil.sh`.
### 3. Sub-pixel blending was wrong for this display
Enabling it made things worse, not better — see the rule at the top. It is off
by default everywhere. Vegas mode used to opt in; it now scrolls in whole
pixels locked to the refresh like the plugin tickers, and keeps the blend only
behind `display.vegas_scroll.sub_pixel_blend` (default `false`). The blend is
also why text looked anti-aliased in the web preview while the panel shimmered.
by default and only Vegas mode opts in via `set_sub_pixel_scrolling(True)`.
### 4. Frame-based stepping raced the vsync clock
@@ -234,9 +231,6 @@ advances by elapsed time at `scroll_speed / scroll_delay` px/s.
## Diagnosing a juddery scroller
To check a whole rig rather than one scroller, soak it -- see *Soaking a rig*
below.
**An average will lie to you.** A 2 ms duplicate frame and a 21 ms double-wait
mean exactly 10 ms, so a ticker stalling on half its frames still averages to a
healthy 100 fps. The stats line reports the tail for that reason — read the
@@ -309,269 +303,6 @@ journalctl -u ledmatrix --since "-5min" --no-pager | grep -iE "px/s|px/frame"
If a plugin logs its scroll config **twice** with different modes, the second
line is what is running.
## Soaking a rig
The per-scroller lines above tell you *which* scroller misbehaves. The soak
answers the question a release has to answer for each rig: **over a long run,
how often did a moving frame reach the panel late?**
Every frame reaches the panel through `DisplayManager.update_display`, so it is
timed there once, whoever drew it -- Vegas, a ticker plugin, anything. The
render thread only appends a tuple; a worker thread aggregates and rewrites
`/dev/shm/ledmatrix_frame_stats.json` every 10 seconds (RAM, so no SD-card
wear). `src/common/frame_timing.py` has the details.
```bash
python3 scripts/frame_soak.py # 10 minutes, as the display is now
python3 scripts/frame_soak.py --preview # with the web preview open
python3 scripts/frame_soak.py --show # totals since the service started
python3 scripts/frame_soak.py --json a.json # keep the report to compare later
```
It runs as any user next to the display service and stops nothing. It needs
something to *scroll* during the run: a live game holding a static scoreboard
on screen gives no verdict. `--preview` keeps the web preview's viewer marker
fresh, which puts the preview's PNG encoding at full rate -- run it as the web
service's user.
| line | what it tells you |
|---|---|
| **Late frames** | Frames presented one or more refreshes after they were due: the panel showed the previous frame again, a visible hitch. **The pass/fail number**, 0.1% by default (`--max-late-pct`). Only intervals between two scrolling frames count, and a frame held for `frame_hold` refreshes is due `frame_hold` refreshes after the last. |
| **Freezes** | Gaps of 250 ms or more inside a scroll: recomposes, plugin handovers, blocking calls on the render thread. Reported but not failed on, because some are handovers between plugins rather than faults. A gap still counts when the display's scroll state went missing for one frame across it, as long as scrolling resumes within 1 s: both of that frame's intervals count. Two static frames in a row end the scroll. (The state expires after 2 s without scroll activity, and plugins can clear it from their own `display()`.) The late and early rates are over frames judged against a known refresh period, which the recorder adopts once two windows in a row agree on it. |
| **blit** | Copying the frame into the matrix canvas (`SetImage`). It grows with width × height × `pwm_bits`: ~5.5 ms at 512×64 with 8 bits on a Pi 4. It is the biggest fixed cost, and it sets the refresh rates a rig can hold one pixel per refresh at. |
| **wait** | Time blocked in `SwapOnVSync`, i.e. the slack left in each refresh. A p50 near zero means the rig has no headroom and anything extra lands a frame late. |
| **work** | Everything else between two frames: drawing, scrolling, and waiting for the GIL. A wide gap between its p50 and p99 is another thread getting in the way. |
| **Binding** | `STOCK` means the rgbmatrix binding holds the GIL through the vsync wait, which starves every other thread. See *Rebuilding the binding*. |
The refresh rate is estimated from the frames themselves (swaps that block on
vsync can only land on refresh boundaries). Cross-check it with
`scroll_speeds.py --measure` if it looks wrong. It can read high on a rig where
nothing ever presented at the full refresh rate.
A soak is only meaningful against a fixed workload. Compare runs with the same
content and `--preview` setting, and alternate which build goes first when you
A/B two of them. A live-API workload drifts over time.
The soak says how often; the service's log says why. A scroll that presents no
frame for 250 ms logs `Render stall:` with the stack of the render thread and
the top of every other thread's, and whether the whole interpreter was blocked
(C code holding the GIL) rather than one thread. To see what is behind the
shorter hitches, run the service with `LEDMATRIX_STALL_WATCHDOG_MS=30`, which
dumps at three refreshes late instead: its extra polling costs a little GIL
time of its own, so do that on a diagnostic run, not a soak you are grading.
`LEDMATRIX_STALL_WATCHDOG=0` turns it off.
### Results: hdpi, 2026-09-24
Pi 4, 4×128×64 on one chain (512×64), `gpio_slowdown` 3, cap 120 Hz, the
GIL-releasing binding. Vegas mode with live content, 8-minute soaks with
`--preview`, run in the order shown so each build went both first and last.
| run | build | pacing | pwm_bits | refresh | late | 1 | 2 | 3–5 | 6+ | freezes |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | main | time-based, blended, 90 px/s | 8 | 94.5 Hz | 6.33% | 2,542 | 74 | 19 | 4 | 0 |
| 2 | #628 | 1 px / refresh | 8 | 100.2 Hz | 0.66% | 238 | 32 | 30 | 5 | 2 |
| 3 | #628 | 1 px / refresh | 8 | 100.3 Hz | 0.70% | 252 | 38 | 26 | 6 | 2 |
| 4 | main | time-based, blended, 90 px/s | 8 | 94.5 Hz | 6.46% | 2,659 | 90 | 10 | 4 | 0 |
| 5 | #628 | 1 px / 2 refreshes (53 px/s) | **7** | 107.2 Hz | 0.32% | 68 | 7 | 4 | 2 | 1 |
- Blending cost the panel refresh rate as well as frames: 94.5 Hz against
~100 Hz for the same hardware under whole-pixel pacing.
- The freezes and the 3+ rows in the #628 runs line up with canvas-bound
plugins fetched on the render thread (`drain_deferred`): `news` took ~320 ms
and `hockey-scoreboard` ~660 ms there. Moving those
fetches off the render thread is proposed separately (offscreen rendering).
- Run 5 changed two things at once: the speed, and `pwm_bits` (changed on the
rig between runs). Its lower late rate cannot be credited to either alone.
- These soaks were taken before the recorder counted 1–2 s stalls as freezes,
so a stall of that length would be missing from these rows.
### Without the service: `render_bench.py`
The soak measures the service as it really runs: live content, plugin
updates, the web preview. `scripts/render_bench.py` answers the narrower
question underneath: *with nothing else in the way, can this hardware present
every frame on time?* It scrolls a synthetic strip through the production path
-- a real `DisplayManager`, a real `ScrollHelper`, the same `scroll_config`
resolver every ticker uses -- on content that is identical every run, which
makes it the tool for comparing rigs (a Pi 3 against a Pi 4, one HAT against
another) and for A/B testing a change to the render path.
```bash
sudo systemctl stop ledmatrix # the service owns the GPIO
sudo python3 scripts/render_bench.py # 60s at one pixel per refresh
sudo python3 scripts/render_bench.py --seconds 600 # the shipping gate
sudo python3 scripts/render_bench.py --speed 50 # a held (frame_hold 2) speed
sudo python3 scripts/render_bench.py --busy 2 # with threads imitating plugin updates
sudo python3 scripts/render_bench.py --json /tmp/pi4-512x64.json
sudo systemctl start ledmatrix
```
It never starts or stops the service itself, so a crash in it cannot leave
the panel dark. It grades with the same recorder as the soak and prints the
same report, with the same exit status, except that **2** also means the run
could not be set up at all (no root, no panel, a fallback display), so a rig
that was never measured cannot pass by accident.
Two differences from the soak matter:
- **It measures the panel first.** Before scrolling it times bare swaps for a
few seconds to get the idle refresh rate, and seeds the recorder with it.
That is what catches a loop that never locked to the panel at all. The first
version of the bench announced its scrolling state once instead of every
frame; the state expired, the dirty-tracking skip fired mid-scroll, and the
loop free-ran at 827 fps. Graded against its own frames that looks perfectly
steady; graded against the panel's measured rate every frame is early, and
the run fails as NOT LOCKED. (The soak has no idle measurement, so it checks
the rate against `limit_refresh_rate_hz` instead: a "refresh" faster than
the cap cannot have been waiting for the panel.)
- **The stall watchdog prints to the terminal.** A frame held up for more than
250 ms prints the stack of what held it up, in the middle of the run.
Measured with the first version of the bench on hdpi (Pi 4, 512x64,
`pwm_bits` 8), two-minute runs at one pixel per refresh: 8 of 11,449 frames
late (0.070%), and with `--busy 2` 3 of 11,445 (0.026%). The render path and
the hardware pass on their own. Compare the soak results above, from the same
rig with the service running, for how much of the late rate comes from
everything else.
### The panel is slower while you are rendering into it
The bench prints two refresh rates, and they differ:
| | Pi 4, 512x64, `pwm_bits` 8 |
|---|---|
| idle, timing bare swaps | 100.4 Hz |
| while scrolling | 96.3 Hz |
Both are real. Driving an LED matrix is bit-banging on the same machine, so
`SetImage` over a 512x64 chain contends with the refresh itself and slows it.
The recorder therefore reads the rendering rate back from the frames: swaps
that block on vsync can only return on a refresh boundary, so the low end of
`interval / frame_hold` is the period. The idle figure is still printed,
because the gap between the two is itself a measure of how expensive a frame
is: **a rise in that gap is a render-cost regression even when nothing is
late.**
The practical consequence for config: set `limit_refresh_rate_hz` near the rate
the panel holds *while rendering*, not the idle rate and certainly not a cap it
can never reach. A cap well above the real rate makes `scroll_config` solve
speeds against a refresh that does not exist, which is where "3px every 4
refreshes" comes from.
### Bench-only counters
| line | meaning |
|---|---|
| `duplicate` | frames that advanced no pixels. A crisp fixed-step scroll should show none; any at all means the loop is presenting faster than the strip is moving. |
| `blank` | frames with no visible slice to draw: the helper had no content. Should be zero. |
| `restarts` | how many times the strip was scrolled through end to end. Informational: the bench restarts the strip where a plugin would hand over to the next one. |
`--json` writes the full report plus the panel geometry, the solved speed and
these counters, so two rigs (or one rig before and after a change) can be
compared without re-reading a terminal.
---
## A tear across the middle on fast scrolls
**Symptom:** while text scrolls, the top and bottom halves of the panel look
shifted sideways against each other along a horizontal line at mid-height, and
the shift grows with scroll speed. It shows most in Vegas mode at high speed.
**It is the panel's scan, not the software.** The measured panel, like most
64-row panels, is multiplexed 1:32 (some panels of the same size scan
differently, so check yours): it lights two rows at a time, one from each half
(row 0 with row 32, row 1 with row 33, …), stepping down both halves together
once per refresh. So row 31,
the last row of the top half, lights almost a whole refresh period after row 32
right below it. Your eye follows moving text, and moving content that lights at
different times lands in different places, so the two rows meet with an offset
of roughly
```
offset ≈ scroll speed × refresh period
```
Each frame reaches the panel whole (`SwapOnVSync` swaps complete frames between
refreshes); the shift is created inside a single refresh. Other panel heights
show it too, at the point where their two scan halves meet.
On the 2×128×64 chain above, which refreshes at about 130 Hz flat out
(7.7 ms per pass):
| scroll speed | offset at the midline |
|---|---|
| 50 px/s (Vegas default) | ~0.4 px |
| 100 px/s | ~0.8 px |
| 150 px/s | ~1.2 px, plainly visible |
### What the display does about it
At one pixel per refresh, the fastest crisp speed, the step is exactly one
refresh's worth of motion, so it can be cancelled: show one half of the panel
a refresh behind the other -- the half whose row at the seam lights at the
start of each refresh. The two rows either side of the seam then show the same
moment again. What is left is a
lean of one pixel per half from top to bottom, continuous across the panel,
which reads as nothing where the step read as a tear. `DisplayManager` does
this while something scrolls at one frame per refresh
(`display.scan_order_compensation`, `"auto"` by default, `"off"` to disable;
the geometry is in `src/scan_order.py`). The lagging rows come from the
previous frame the display presented, so it works for Vegas and every plugin
ticker without knowing how they scroll.
Checked on hdpi (4×128×64 on one chain, rotated 180, 2026-09-24) before it was
written: `scan_mode: 1` (interlaced) made the step vanish but turned moving
edges grainy, and halving the speed halved it, so it is the scan and not a torn
frame. With the compensation the step is gone at 90 px/s.
It is left off where the row order is unknown or the maths does not hold:
- **Slower speeds**, where each frame is held for two or more refreshes. The
offset there is half a pixel or less, and cancelling it would need a lag of
a fraction of a frame.
- **Other layouts:** pixel mappers other than a 0 or 180 degree rotation
(U-mapper, 90/270), non-zero `multiplexing`, interlaced `scan_mode`, and a
canvas remapped to another height (double-sided mode).
- **The emulator,** which has no scan order.
### When it cannot apply
Only a shorter scan period (a faster refresh) or a slower scroll. Measure what
the panel actually achieves first. The library prints the rate with a carriage
return and no newline, so read it from the raw journal:
```bash
# set display.hardware.show_refresh_rate to true (web UI, Display tab), restart, then:
journalctl -u ledmatrix --since "-1min" --no-pager -o cat --all | grep -a -oE "[0-9.]+Hz" | tail -5
```
Turn it off again afterwards. Measured on that panel (Pi 4, single chain),
changing one setting at a time from `pwm_bits: 7`, `gpio_slowdown: 3`:
| change | refresh, uncapped | notes |
|---|---|---|
| none | ~130 Hz | the ceiling for this wiring |
| `pwm_bits: 6` | ~138 Hz | barely faster, and half the colour depth |
| `gpio_slowdown: 2` | ~130 Hz | no faster, **and visible glitching**; keep 3 |
| `limit_refresh_rate_hz: 0` | ~130 Hz | Vegas dropped from 100 to 72–95 fps as the refresh thread took more CPU |
None of these helps much, because the time goes into shifting each row's pixels
out: a 2×128 chain pushes 256 pixels per row down one output. What does help is
**fewer pixels per output**. On a bonnet with more than one output (the
`regular` and `classic` mappings have 3; `adafruit-hat` has 1), put each panel
on its own output and set `parallel` to the number of outputs used and
`chain_length` to the panels per output, for example `parallel: 2`,
`chain_length: 1` for two panels. Each refresh then shifts half the data, which
should roughly double the refresh rate and halve the offset. That is a cable
change, so measure again afterwards.
Short of rewiring, keep fast scrolls moderate on those layouts: at 50 px/s the
offset is under half a pixel.
## Rebuilding the binding
```bash
+206
View File
@@ -0,0 +1,206 @@
# Skin System Architecture
## Status: not supported yet
**Skins don't render with the current scoreboard plugins.** The skin system
below works in isolation (it loads, validates and renders skins in
`scripts/validate_skin.py` and `test/test_skin_system.py`), but nothing on a
running display calls it:
- The only render hook is `SportsCore._render_game()` in
`src/base_classes/sports/core.py`.
- None of the current scoreboard plugins build on `src.base_classes`. The
official scoreboards in the `ledmatrix-plugins` monorepo, and the
third-party scoreboards in the plugin registry, carry their own sports and
rendering code (with the shared `src/common/sports_*` helpers) and never
reach `SportsCore._render_game()`.
So a skin can be dropped into `skins/` and named in a plugin's config, but the
scoreboard keeps drawing its built-in layout. Until a scoreboard adopts the
hook, core does not offer skins to users:
- The plugin config page shows no **Visual Skin** dropdown.
- The Plugin Store hides registry entries with `"type": "skin"` and refuses
to install one (`POST /api/v3/plugins/install` answers 400 with the reason).
- `GET /api/v3/skins` still lists what is in `skins/`, with
`"supported": false` and a `message`.
- A config that already contains `"skin"` / `"skin_options"` still loads,
validates and saves unchanged; the value is simply unused.
The rest of this document describes the design as built, for whoever wires a
scoreboard to it.
Skins are user-installable **visual overlays** for the sports scoreboards.
A skin replaces only the *look* of a scoreboard — the host plugin keeps doing
data fetching, scheduling, caching, dedup, live-priority takeover, and vegas
mode. If you only want to **build** a skin, read
[CREATING_SKINS.md](CREATING_SKINS.md); this document explains how the system
works and why it is shaped this way.
## Why skins instead of forks
Before skins, changing a scoreboard's layout meant forking the whole plugin
(e.g. the community MLB scoreboard fork). The fork gets the new look but loses
everything the maintained plugin keeps earning: duration/scheduling behavior,
vegas mode support, caching and background-fetch improvements, bug fixes. It
also silently drifts: every upstream improvement now has to be re-ported by
hand.
A skin inverts that trade. The plugin remains stock and keeps updating through
the store; the skin is ~100 lines of pure rendering code that receives the
plugin's already-fetched data each frame. Uninstalling the skin (or the skin
crashing) simply restores the built-in look.
```text
(unchanged) (the skin seam)
ESPN API ──► update() ──► game view model ──► _render_game() ──► display
fetching (a dict) │ │
caching │ └─ built-in
scheduling └─ skin.render_<mode>(ctx, game)
live priority draws onto ctx.canvas
```
## The render funnel
A sports scoreboard built on the `src/base_classes/sports/` package
(`core.py`) renders through exactly one seam. No current scoreboard plugin is
built on it (see [Status](#status-not-supported-yet)), so for them this seam is
never reached:
`SportsCore._render_game(game, force_clear)`.
1. The mode class's `display()` (live, `SportsUpcoming`, `SportsRecent`)
picks `self.current_game` and calls `_render_game`.
2. `_render_game` lazily loads the configured skin (once, on first render —
a broken skin can never block plugin startup).
3. If a skin is active, the host builds a `SkinContext` — a fresh black
canvas at the current display size plus layout/font/logo helpers — and
calls the skin's `render_live` / `render_recent` / `render_upcoming`
with a **copy** of the game dict.
4. If the skin returns `True`, the canvas is composited onto the display.
If it returns `False`, isn't implemented for that mode, or raises, the
built-in `_draw_scorebug_layout` runs instead.
Key properties that fall out of this design:
- **Per-mode fallback.** A skin that only implements `render_live` gets the
stock recent/upcoming screens for free.
- **Three strikes.** A skin that raises 3 times in a row is disabled for the
rest of the session (one loud error log per failure); the display never
goes dark. Restarting the service re-arms it.
- **Copies, not references.** Skins receive a shallow copy of the game dict,
so a buggy skin cannot corrupt the plugin's scheduling state.
- **Vegas mode works untouched.** Vegas capture falls back to grabbing the
regular `display()` output, which is already skin-rendered. Skins can
additionally implement `render_vegas_card` for purpose-built scroll cards,
and hosts can call `SportsCore.render_skin_card(game, size)` to use it.
- **Hot-loop caution.** `render_live` runs every display-loop pass during a
live game. The host logs a warning when a skin render exceeds 150 ms, and
`scripts/validate_skin.py` enforces a budget at development time — but
Python cannot forcibly time-out a stuck render, so a skin that blocks
(network I/O, giant image ops) stalls the display. This is why the rules
in CREATING_SKINS.md ban I/O in render paths.
## The view model contract
The `game` dict a skin receives is the plugin's already-extracted view model
(`SportsCore._extract_game_details_common` plus per-sport extras from
`src/base_classes/{baseball,basketball,football,hockey}.py`).
- **Guaranteed keys (view model v1.0)** — always present for every sport:
`id`, `game_time`, `game_date`, `start_time_utc` (a UTC `datetime`),
`status_text`, `is_live`, `is_final`, `is_upcoming`, `is_halftime`,
`home_abbr`/`away_abbr`, `home_id`/`away_id`, `home_score`/`away_score`
(**strings**), `home_logo_path`/`away_logo_path`, `home_record`/`away_record`.
- **Sport extras** — documented per sport in CREATING_SKINS.md (e.g. baseball
adds `inning`, `inning_half`, `balls`, `strikes`, `outs`, `bases_occupied`).
- **Optional keys** (`odds`, rankings, `series_summary`, …) are present only
when the feature is enabled — skins must always use `.get()`.
Versioning policy: additive changes bump the minor version
(`VIEW_MODEL_VERSION` in `src/skin_system/skin_base.py`, surfaced to skins as
`ctx.view_model_version`); renaming or removing a guaranteed key requires a
major bump plus a compat shim. `test/test_skin_system.py::TestViewModelContract`
fails CI if a guaranteed key disappears from the extractor.
Separately, `SKIN_API_VERSION` versions the Python API (`ScoreboardSkin`,
`SkinContext`). The loader refuses a skin whose manifest declares a different
major version and falls back to the built-in renderer with a clear
"skin needs an update" log line.
## Package layout and lifecycle
```text
skins/<skin-id>/
skin.json # manifest (required)
skin.py # ScoreboardSkin subclass (required)
preview.png # optional, shown by the web UI
assets/ # optional skin-local images
helpers.py ... # optional extra modules (namespaced per skin at import)
```
Skins live in the central `skins/` directory — deliberately **not** inside the
plugin's directory, because plugin reinstall/update deletes the whole plugin
directory and a skin must survive that. One skin can also target several
plugins (mlb + milb).
Lifecycle: discovered lazily on first render → manifest validated → API major
version gated → module imported under a namespaced `sys.modules` key (two
skins can both ship a `helpers.py`, same scheme plugins use) → instantiated
with `(manifest, options)`. Every failure logs and falls back to built-in.
Skins should be **stateless**: the live, recent, and upcoming mode classes
each hold their own skin instance, so derive everything from `(ctx, game)`.
## Selection and configuration
Inside the plugin's own config section in `config/config.json`:
```json
"baseball-scoreboard": {
"skin": "retro-baseball",
"skin_options": { "accent_color": [255, 80, 0] }
}
```
`"skin"` is either one id for all modes or a per-mode mapping
(`{"live": "retro-baseball", "recent": "built-in"}`). Absent, empty, or
`"built-in"` means the stock renderer. Because this rides the plugin's config
section, it persists across plugin reinstalls like every other setting.
`SchemaManager.inject_skin_selector` can add a **Visual Skin** enum to the
*served* schema for plugins with matching skins installed. While skins are
unsupported the plugin schema endpoint does not call it, so the dropdown is
not shown. Validation never sees the enum either way: the base schema allows
any `skin` value, so a config that references an uninstalled skin stays valid.
`GET /api/v3/skins` lists installed skins (optionally filtered by
`?plugin_id=`) and reports `"supported": false`.
## Distribution
- **Manual:** `git clone <skin repo> skins/<skin-id>` — that's the whole
install. No manifest bumps, no `update_registry.py`; skins are not monorepo
plugins.
- **Store (disabled while unsupported):** registry entries with
`"type": "skin"` are hidden from the store list and refused on install.
`PluginStoreManager._install_skin_from_info` is kept: once
`SKINS_RENDER_SUPPORTED` in `src/skin_system/__init__.py` is true, such
entries install through the same `plugins.json` pipeline, land in `skins/`,
are validated against `skin.json` (including the API major version) instead
of `manifest.json`, and never install dependencies — skins are render-only
(stdlib + PIL + the provided context, no third-party packages in v1).
## Trust model
A skin is Python executing inside the display service — **exactly the same
trust level as a plugin**, even though "skin" sounds cosmetic. Only install
skins from sources you'd be willing to install a plugin from.
## v2 directions (not in v1)
- A generic `BasePlugin` opt-in (`render_with_skin()`) so non-sports plugins
(weather, music) can offer skinnable layouts; `skin_runtime` is already
sports-agnostic in anticipation.
- Store UI: preview gallery, one-click install from the skin browser.
- An update path for git-cloned skins (today: re-clone or store reinstall).
- Animation support in skins (today the API is one frame per render call;
stateful tricks work but are at-your-own-risk).
+34 -32
View File
@@ -7,9 +7,9 @@ becoming nine clients of a god class.
Nine plugins (`afl`, `baseball`, `basketball`, `football`, `hockey`, `lacrosse`,
`nrl`, `soccer`, `ufc`) each ship a ~3,000-line `sports.py` descended from this
repo's former `src/base_classes/sports.py` (since removed). They have drifted
into three lineages, and only 28 of the 66 methods appearing across them are
present in all nine. One logical fix (the UTC start-time bug) cost 75 files.
repo's `src/base_classes/sports.py`. They have drifted into three lineages, and
only 28 of the 66 methods appearing across them are present in all nine. One
logical fix (the UTC start-time bug) cost 75 files.
Merging everything into one base class would fix the duplication and create a
worse problem: a single 2,500-line class that all nine plugins inherit, where any
@@ -26,7 +26,7 @@ These are independent concerns. Conflating them is what produces god classes.
|---|---|
| Plugin loads on a core that predates a module | Guarded import with a bundled fallback (`try: from src.X import Y / except ModuleNotFoundError: from y import Y`) |
| Plugin loads on a core that predates a *method* | Capability probing — `hasattr(SportsCore, "_detect_stale_games")` — never a version comparison. The loader's compat check is advisory-only (it logs and continues), so probing is the real protection. |
| Core changes never break a plugin's rendering | The **view-model contract**: the game dict each plugin's `_extract_game_details_common` builds is read by the shared `src/common` renderers, so its keys may be added, never renamed or removed. |
| Core changes never break a plugin's rendering | The **view-model contract**: `_extract_game_details_common` returns a dict whose `GUARANTEED_KEYS` are frozen by `test/test_skin_system.py::TestViewModelContract`. Keys may be added, never renamed or removed. |
| A plugin can drop its bundled copy safely | The **sunset rule**: its manifest must floor `ledmatrix_min_version` at the first core release shipping the module (recorded in `CHANGELOG.md`) — *necessary but not sufficient*. The store enforces that floor on every registry-managed install and on both supported update paths (sideloading via `install_from_url` is not gated), but a floor cannot reach a user who never updates, so the copy also waits for the B6 gate below. |
The core API is **additive-only**. A method the plugins call is never removed or
@@ -67,38 +67,40 @@ This is the property the naive merge destroys, and it is enforced structurally:
## Layering
`src/base_classes/` has been removed: no scoreboard plugin built on it. B1 and
B2 below promoted code into it (`SportsCore`, the mode classes,
`CelebrationMixin`, the rotation strategies); the override points and
capabilities sections record that design, but none of it ships in core any
more. Shared sports code lives in `src/common`:
```
src/base_classes/sports/
__init__.py re-exports the public API (import path unchanged)
core.py SportsCore — fetch, cache, config, logos, fonts, odds,
view-model extraction, the skin seam
modes.py SportsUpcoming / SportsRecent / SportsLive
capabilities/
celebrations.py CelebrationMixin (opt-in: 4 of 9 plugins)
rotation.py RotationStrategy + registry
| Module | Since | Holds |
|---|---|---|
| `sports_scroll.py` | 3.2.0 | `SportsScrollDisplay` / `SportsScrollDisplayManager` — scroll orchestration (content building stays in the plugins) |
| `sports_card.py` | 3.3.0 | Free functions for card settings, colours, favourite-team rules, dates and font sizes |
| `sports_game_renderer.py` | 3.3.0 | `SportsGameRendererMixin` — scroll/Vegas card geometry |
| `sports_shared.py` | 3.3.0 | `SportsCoreSharedMixin`, `SportsLiveSharedMixin`, `SportsRecentSharedMixin` — the sport-independent `sports.py` methods |
| `sports_helpers.py` | 3.5.0 | clamp/logo/rotation free functions and `SportsHelpersMixin`, plus the `_favorite_key` seam |
| `espn_dates.py` | 3.5.0 | ESPN date-range and `limit` workarounds |
| `favorite_team_check.py` | 3.6.0 | `FavoriteTeamCheck` — logs why a favourite team code shows nothing |
| `sports_timezone.py` | 3.6.0 | Which timezone start times are drawn in (`resolve_timezone_name`) |
src/common/
sports_scroll.py SportsScrollDisplay / …Manager — scroll orchestration
(content building stays in the plugins)
sports_helpers.py clamp/logo/rotation free functions + SportsHelpersMixin
(3.5.0) — the helpers byte-identical in the
plugins' sports.py, and the _favorite_key seam
```
Each is described in [src/common/README.md](../src/common/README.md).
`from src.base_classes.sports import SportsCore` keeps working — the package
`__init__` re-exports, so the conversion is invisible to every existing importer.
### Converging on `src/common`
The scoreboards never built on `src/base_classes` (now removed); their own
`sports.py` copies had moved past it. So shared code now lands in hardware-free `src/common`
The scoreboards do not build on `src/base_classes`; their own `sports.py` copies
have moved past it. So shared code now lands in hardware-free `src/common`
modules taken from the plugin copies, each a **new module** rather than growth
on an existing one: a plugin that deletes a method copy and relies on an older
module having gained it fails at runtime with an `AttributeError`, while a
missing module fails at load, where the version checks can see it.
`sports_helpers.py` is the newest (it holds `_favorite_key`, the override point
`sports_helpers.py` is the first (it holds `_favorite_key`, the override point
listed below, for later phases); its parity test compares every body against
the plugin copies when `LEDMATRIX_PLUGINS` points at a checkout, and
`test/test_common_is_hardware_free.py` keeps `src/common` free of
`rgbmatrix`, `src.display_manager` and `src.plugin_system`. How a plugin adopts a
`rgbmatrix`, `src.base_classes` and `src.plugin_system`. How a plugin adopts a
module and drops its copy is documented in the plugins repo's
`docs/plugin-development/08-shared-sports-code.md`.
@@ -114,6 +116,7 @@ deprecation cycle.
| `_extract_game_details(event)` | Sport-specific view-model fields on top of the common ones | delegates to `_extract_game_details_common` |
| `_draw_scorebug_layout(game, force_clear)` | Sport's card rendering | base layout |
| `_custom_scorebug_layout(game, draw)` | Per-sport overlay on the base layout | no-op |
| `render_skin_card(game, size)` | Skin-system entry point | built-in fallback |
| `score_phrase(points, team_abbr)` | Celebration wording (`"GOOOOAAALLL!"` vs `"TOUCHDOWN!"`). `points` is the score delta, which sports with variable-value scores use to name the play | `"<abbr> SCORES!"` — only consulted when `CelebrationMixin` is present |
| `win_phrase(team_abbr)` | Win-celebration wording | `"<abbr> WINS!"` — mixin only |
| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching | `game["<side>_abbr"]` |
@@ -186,9 +189,9 @@ and a typo should cost the boost, not the scoreboard. When a plugin needs an
ordering that core does not ship, it calls `register_rotation_strategy` to add
its own — rather than core growing a branch for it.
`test_sports_capabilities.py` (removed with `src/base_classes`) checked each
strategy against a **verbatim transcription** of the plugin code it replaces,
over every live-game shape up to four games. That differential is what B5 deletes the bundled copies on the
`test_sports_capabilities.py` checks each strategy against a **verbatim
transcription** of the plugin code it replaces, over every live-game shape up to
four games. That differential is what B5 deletes the bundled copies on the
strength of.
## Scroll display — where the promotion line falls
@@ -504,9 +507,9 @@ What actually remains, smallest first:
3. **Reconsider the held modules** (`data_sources.py`, `game_renderer.py`,
`base_odds_manager.py`) now that the sunset has closed. `game_renderer.py` is
the largest single duplication left: ~11,500 lines across eight plugins, with
~36,500 more in the eight `sports.py`. The `src/base_classes/sports/`
package promoted in B1/B2 was never imported by a plugin and has been
removed, so the plugin copies are the only starting point.
~36,500 more in the eight `sports.py`. Note that core already ships
`src/base_classes/sports/` (~143KB, promoted in B1/B2) that **no plugin
imports** — check whether it has drifted before treating it as the target.
## How to keep this project healthy
@@ -539,7 +542,6 @@ Lessons this migration paid for, worth applying beyond it:
- **A capability that is not opted into must not execute.** If you find yourself
writing `if self.<capability>_enabled` inside a base class, it belongs in a
mixin.
- **Touch the view-model keys only additively.** The shared `src/common`
renderers read them.
- **Touch the view-model keys only additively.** Published skins depend on them.
- **Every promotion lands with the characterization suite green**, and every
pilot adoption lands with that plugin's harness and golden suites green.
+8 -18
View File
@@ -102,16 +102,9 @@ cd /path/to/LEDMatrix
bash scripts/download_pixlet.sh
```
The script downloads only the Linux ARM64 build (Raspberry Pi OS 64-bit),
from the `tronbyt/pixlet` releases, to `bin/pixlet/pixlet-linux-arm64`. On
any other platform (32-bit Pi OS, x86_64, macOS), put a `pixlet` binary on
your `PATH`, or set the plugin's `pixlet_path`, or place it in `bin/pixlet/`
under the name `_find_pixlet_binary()` looks for
([`web_interface/blueprints/api_v3/__init__.py`](../web_interface/blueprints/api_v3/__init__.py)).
Verify installation:
```bash
./bin/pixlet/pixlet-linux-arm64 version
./bin/pixlet/pixlet-linux-amd64 version
# Pixlet 0.50.2 (or later)
```
@@ -134,7 +127,7 @@ Verify installation:
3. Filter by category: Weather, Sports, Finance, Games, Clocks, etc.
4. Click **Install** on desired apps
5. Configure each app:
- Set location/timezone (optional: blank uses this device's location)
- Set location/timezone
- Enter API keys if required
- Customize display preferences
@@ -144,12 +137,7 @@ Each app may have different configuration options:
#### Common Configuration Types
- **Location** (lat/lng/timezone): For weather, clocks, transit. Left blank,
the app renders at this device's location (City / State / Country under
General settings). If no city is set there, or the city can't be looked up
(no match, or the geocoder is unreachable -- retried after 30 minutes), the
app gets no location and falls back to its author's hard-coded default,
usually San Francisco. Fill it in only to point one app somewhere else.
- **Location** (lat/lng/timezone): For weather, clocks, transit
- **API Keys**: For services like weather, stocks, sports scores
- **Display Preferences**: Colors, units, layouts
- **Dropdown Options**: Team selections, language, themes
@@ -283,8 +271,10 @@ LEDMatrix/
│ ├── hour_hand.png
│ └── minute_hand.png
│
├── bin/pixlet/ # Pixlet binary
│ └── pixlet-linux-arm64 # the only one download_pixlet.sh fetches
├── bin/pixlet/ # Pixlet binaries
│ ├── pixlet-linux-amd64
│ ├── pixlet-linux-arm64
│ └── pixlet-darwin-arm64
│
└── scripts/
└── download_pixlet.sh # Pixlet installer
@@ -329,7 +319,7 @@ Many apps require API keys for external services:
**Solutions**:
1. Check logs: `journalctl -u ledmatrix | grep -i pixlet`
2. Verify config: Ensure all required fields are filled
3. Test manually: `./bin/pixlet/pixlet-linux-arm64 render starlark-apps/{app-id}/{app-id}.star`
3. Test manually: `./bin/pixlet/pixlet-linux-amd64 render starlark-apps/{app-id}/{app-id}.star`
4. Missing assets: Some apps need images/fonts that may fail to download
5. API issues: Check API keys and rate limits
+27 -41
View File
@@ -201,11 +201,10 @@ sudo systemctl restart ledmatrix-web
**Solutions:**
1. **Install dependencies** as root, so the root display service can import
them:
1. **Install dependencies:**
```bash
sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
sudo python3 -m pip install --break-system-packages --no-cache-dir -r web_interface/requirements.txt
pip3 install --break-system-packages -r requirements.txt
pip3 install --break-system-packages -r web_interface/requirements.txt
```
2. **Test imports step-by-step:**
@@ -251,18 +250,15 @@ sudo systemctl restart ledmatrix-web
**Solutions:**
[PERMISSIONS.md](PERMISSIONS.md) lists the expected owner and mode of every
file and directory, and which `scripts/fix_perms/` script to run as which
user. Don't `chown -R` the whole project: the two sudo helper scripts in
`scripts/fix_perms/` must stay owned by root.
```bash
# Config files: web user owns both; secrets must stay 640
stat -c '%U:%G %a %n' config/config.json config/config_secrets.json
# Fix ownership of LEDMatrix directory
sudo chown -R ledpi:ledpi /home/ledpi/LEDMatrix
# Fix config file permissions
sudo chmod 644 config/config.json
sudo chmod 640 config/config_secrets.json
# Which user the web interface runs as
# Verify service runs as correct user
sudo systemctl cat ledmatrix-web | grep User
```
@@ -466,17 +462,15 @@ sudo systemctl cat ledmatrix-web | grep User
}
```
Or toggle the plugin on in the **Plugin Manager** tab, which writes the
same flag.
2. **Restart display:**
```bash
sudo systemctl restart ledmatrix
```
2. **Wait a few seconds.** The display service watches `config.json` and
loads a newly enabled plugin without a restart
(`DisplayController._reconcile_enabled_plugins()` in
[`src/display_controller.py`](../src/display_controller.py)). This
needs hot reload, which is on unless `LEDMATRIX_HOT_RELOAD=false` is set.
3. **If it still does not appear**, check the logs for a config validation
error, then restart: `sudo systemctl restart ledmatrix`
3. **Verify in web interface:**
- Open the **Plugin Manager** tab
- Toggle the plugin switch to enable
- From **Overview**, click **Restart Display Service**
#### Plugin Not Loading
@@ -497,12 +491,10 @@ sudo systemctl cat ledmatrix-web | grep User
# Verify all required fields present
```
3. **Check dependencies installed.** Install them with `sudo`: the display
service runs as root and does not see packages pip put in your user's
`~/.local` (see [PLUGIN_DEPENDENCY_GUIDE.md](PLUGIN_DEPENDENCY_GUIDE.md)):
3. **Check dependencies installed:**
```bash
if [ -f plugin-repos/plugin-id/requirements.txt ]; then
sudo python3 -m pip install --break-system-packages --no-cache-dir -r plugin-repos/plugin-id/requirements.txt
pip3 install --break-system-packages -r plugin-repos/plugin-id/requirements.txt
fi
```
@@ -511,9 +503,14 @@ sudo systemctl cat ledmatrix-web | grep User
sudo journalctl -u ledmatrix -f | grep plugin-id
```
5. **Load and render the plugin headlessly:**
5. **Test plugin import:**
```bash
python3 scripts/check_plugin.py --plugin plugin-id
python3 -c "
import sys
sys.path.insert(0, 'plugin-repos/plugin-id')
from manager import PluginClass
print('Plugin imports successfully')
"
```
#### Stale Cache Data
@@ -543,12 +540,10 @@ sudo systemctl cat ledmatrix-web | grep User
sudo systemctl restart ledmatrix
```
2. **Check cache permissions.** Expected: `root:ledmatrix`, `drwxrwsr-x`.
`setup_cache.sh` restores that layout (see
[PERMISSIONS.md](PERMISSIONS.md#repair-scripts)):
2. **Check cache permissions:**
```bash
ls -ld /var/cache/ledmatrix
sudo bash scripts/install/setup_cache.sh
sudo ./scripts/fix_perms/fix_cache_permissions.sh
```
---
@@ -616,15 +611,6 @@ sudo systemctl cat ledmatrix-web | grep User
```
**Note:** Minimum recommended: 300 seconds (5 minutes)
How often the core calls the plugin's `update()` comes from the plugin
itself first: its `get_update_interval()` if it has one, then
`update_interval` in its `manifest.json`. The `update_interval` in
`config.json` is used by the scheduler only when the manifest sets none.
Many plugins also read their own config `update_interval` and skip the
API call inside `update()` until it has elapsed, which is what makes the
setting above effective; check the plugin's settings form or
`config_schema.json` for the option it actually honours.
2. **Check current rate limit usage:**
- OpenWeatherMap free tier: 1,000 calls/day, 60 calls/minute
- With 300s interval: 288 calls/day (well within limits)
+5 -9
View File
@@ -161,11 +161,7 @@ duration, and related settings — so you can configure Vegas mode
entirely from the web UI without hand-editing JSON. See
[ADVANCED_FEATURES.md](ADVANCED_FEATURES.md) for what the options do.
Brightness and the Vegas Scroll settings apply to the running display
within a few seconds. Matrix hardware settings (rows, columns, chain length,
mapping, GPIO slowdown, PWM and refresh settings) are only read when the
display starts, so those need **Restart Display Service** from the Overview
tab.
Changes require **Restart Display Service** from the Overview tab.
### Plugin Manager Tab
@@ -252,16 +248,16 @@ View real-time system logs:
1. Open the **Display** tab
2. Adjust the **Brightness** slider (1–100)
3. Click **Save**. The panel picks up the new brightness within a few
seconds; no restart is needed
3. Click **Save**
4. Click **Restart Display Service** on the **Overview** tab
### Installing a New Plugin
1. Open the **Plugin Manager** tab
2. Scroll to the **Plugin Store** section and browse or search
3. Click **Install** next to the plugin
4. Toggle the plugin on in **Installed Plugins**. The running display
loads it within a few seconds; no restart is needed
4. Toggle the plugin on in **Installed Plugins**
5. Click **Restart Display Service** on **Overview**
### Configuring a Plugin
+588 -5
View File
@@ -1,7 +1,590 @@
# Widget Development Guide
The widget guide lives next to the widgets, in
[web_interface/static/v3/js/widgets/README.md](../web_interface/static/v3/js/widgets/README.md).
It lists every built-in `x-widget`, the schema keywords the config form
understands (`x-options.labels`, `x-advanced`, `x-display: "hidden"`), and how
to ship a custom widget with a plugin.
## Overview
The LEDMatrix Widget Registry system allows plugins to use reusable UI components for configuration forms. This enables:
- **Reusable Components**: Use existing widgets (file upload, checkboxes, etc.) without custom code
- **Custom Widgets**: Create plugin-specific widgets without modifying the LEDMatrix codebase
- **Backwards Compatibility**: Existing plugins continue to work without changes
## Available Core Widgets
### Plugin File Manager Widget (`plugin-file-manager`)
Full inline file management UI for plugins that manage files via the `web_ui_actions` system. Renders a card grid, upload zone, create/delete modals, and an entry table editor — entirely inline, no iframe.
`plugin_id` is **automatically injected** from template context. File operations call `/api/v3/plugins/action` immediately on user action; no Save Configuration needed.
**Schema Configuration:**
```json
{
"file_manager": {
"type": "null",
"title": "Data Files",
"x-widget": "plugin-file-manager",
"x-widget-config": {
"actions": {
"list": "list-files",
"get": "get-file",
"save": "save-file",
"upload": "upload-file",
"delete": "delete-file",
"create": "create-file",
"toggle": "toggle-category"
},
"upload_hint": "JSON files with day numbers 1–365 as keys",
"directory_label": "my_data/",
"create_fields": [
{ "key": "category_name", "label": "Category Name",
"placeholder": "e.g., my_words", "pattern": "^[a-z0-9_]+$",
"hint": "Lowercase letters, numbers, underscores" },
{ "key": "display_name", "label": "Display Name",
"placeholder": "e.g., My Words", "hint": "Optional" }
]
}
}
}
```
**`list` is required** — the widget calls it on render to populate the file grid; omitting it leaves the widget stuck in a loading state. All other actions are optional — omit any key to hide its UI element (e.g., no `create` = no New File button, no `toggle` = no enable/disable switch).
The edit view auto-detects whether file content is tabular (object-of-objects with uniform keys) and shows a paginated table editor with inline cells. Otherwise falls back to a JSON textarea.
**Used by:** of-the-day
---
### Time Picker Widget (`time-picker`)
Single time selection using the browser's native time input. Returns a string in `HH:MM` (24-hour) format. Generic — works in any plugin without configuration.
**Schema Configuration:**
```json
{
"target_time": {
"type": "string",
"x-widget": "time-picker",
"default": "00:00",
"x-options": {
"placeholder": "Select time",
"clearable": true
}
}
}
```
**Used by:** countdown
---
### File Upload Single Widget (`file-upload-single`)
Single-image upload for string fields. Uploads to the plugin's asset folder (`assets/plugins/<plugin_id>/uploads/`) and sets the string field value to the returned relative path. Shows a thumbnail preview and a clear button. The `plugin_id` is **automatically injected** from the template context — no need to specify it in the schema.
**Schema Configuration:**
```json
{
"image_path": {
"type": "string",
"x-widget": "file-upload-single",
"x-upload-config": {
"allowed_types": ["image/png", "image/jpeg", "image/bmp", "image/gif"],
"max_size_mb": 5
}
}
}
```
Note: Unlike `file-upload` (array-level), this widget is for a single `string` field. It is ideal for per-item images inside `array-table` rows.
**Used by:** countdown
---
### File Upload Widget (`file-upload`)
Upload and manage image files with drag-and-drop support, preview, delete, and scheduling.
**Schema Configuration:**
```json
{
"type": "array",
"x-widget": "file-upload",
"x-upload-config": {
"plugin_id": "my-plugin",
"max_files": 10,
"max_size_mb": 5,
"allowed_types": ["image/png", "image/jpeg", "image/bmp", "image/gif"]
}
}
```
**Used by:** static-image, news plugins
### Checkbox Group Widget (`checkbox-group`)
Multi-select checkboxes for array fields with enum items.
**Schema Configuration:**
```json
{
"type": "array",
"x-widget": "checkbox-group",
"items": {
"type": "string",
"enum": ["option1", "option2", "option3"]
},
"x-options": {
"labels": {
"option1": "Option 1 Label",
"option2": "Option 2 Label"
}
}
}
```
**Used by:** odds-ticker, news plugins
### Custom Feeds Widget (`custom-feeds`)
Table-based RSS feed editor with logo uploads.
**Schema Configuration:**
```json
{
"type": "array",
"x-widget": "custom-feeds",
"items": {
"type": "object",
"properties": {
"name": { "type": "string" },
"url": { "type": "string", "format": "uri" },
"enabled": { "type": "boolean" },
"logo": { "type": "object" }
}
},
"maxItems": 50
}
```
**Used by:** news plugin (for custom RSS feeds)
## Using Existing Widgets
To use an existing widget in your plugin's `config_schema.json`, simply add the `x-widget` property:
```json
{
"properties": {
"my_images": {
"type": "array",
"x-widget": "file-upload",
"x-upload-config": {
"plugin_id": "my-plugin",
"max_files": 5
}
},
"enabled_leagues": {
"type": "array",
"x-widget": "checkbox-group",
"items": {
"type": "string",
"enum": ["nfl", "nba", "mlb"]
},
"x-options": {
"labels": {
"nfl": "NFL",
"nba": "NBA",
"mlb": "MLB"
}
}
}
}
}
```
The widget will be automatically rendered when the plugin configuration form is loaded.
## Labelling Enum Options (`x-options.labels`)
A plain `enum` renders as a dropdown whose option text is the value with
underscores replaced and title case applied — `day_first` becomes "Day First".
That is fine for values that read as their own label, and wrong for values that
do not: `vs` becomes "Vs", and `abbrev` says nothing about the `Sep 19` it
actually produces.
Supply `x-options.labels` to set the visible text. This is the same convention
the `checkbox-group` widget uses:
```json
{
"date_format": {
"type": "string",
"enum": ["abbrev", "numeric", "day_first"],
"default": "abbrev",
"x-options": {
"labels": {
"abbrev": "Sep 19",
"numeric": "9/19",
"day_first": "19 Sep"
}
}
}
}
```
Labels are **display only** — the stored value is still the enum value, so
adding them never changes a saved config. The map may be partial: any value
without a label keeps the humanised fallback. Older cores that predate this
support ignore `x-options` and render the fallback for every option, so a
plugin can ship labels without requiring a core upgrade.
Array-table columns (`x-widget: array-table`) accept the same
`x-options.labels` on a column definition, but their fallback is the **raw
value** rather than the humanised one, because those columns hold values such
as ticker symbols where `aapl` → "Aapl" would be wrong. Rows added in the
browser use the labels too (`array-table.js`), so a column reads the same
before and after a page reload.
## Marking Fields as Advanced (`x-advanced`)
Add `"x-advanced": true` to any top-level, non-object property to move it out
of the main form and into a single collapsed **Advanced Settings** section at
the bottom of the plugin's configuration page:
```json
{
"properties": {
"city": {
"type": "string",
"title": "City"
},
"request_timeout": {
"type": "integer",
"default": 10,
"description": "HTTP timeout in seconds",
"x-advanced": true
}
}
}
```
Guidelines:
- Use it for fine-tuning knobs most users never touch (timeouts, retry
behavior, cache TTLs, styling overrides). Anything a first-time user must
set to get the plugin working should stay basic.
- Nothing is hidden permanently — the section expands on click, and the
settings search finds and auto-expands advanced fields like any others.
- The flag is ignored on `object`-type properties (they already render as
their own collapsible sections) and is safely ignored by older cores, so
adding it never breaks compatibility.
## Hiding Fields From the Form (`x-display: "hidden"`)
Add `"x-display": "hidden"` to a property that must stay in the schema but
should not appear as a control: a deprecated key kept so existing configs keep
validating, or an internal value such as an auto-generated row id.
```json
{
"properties": {
"radar_zoom": {
"type": "integer",
"default": 6,
"title": "Radar Zoom Level (deprecated)",
"x-display": "hidden"
}
}
}
```
What the core does with it:
- **Not rendered** at any depth: top-level fields, children of an object
section, and properties of array-of-object items (never a table column, even
if `x-columns` names it, and never in the row editor). A hidden field flagged
`x-advanced` is not listed or counted in Advanced Settings, and an object
whose children are all hidden draws no empty section. Hidden fields don't
show up in the settings search either, since it indexes the rendered form.
- **Stored value preserved on save.** Saving the form never changes a hidden
value. The unchecked-checkbox rule ignores a hidden boolean. Array rows carry
a hidden property's stored value through the form, so the value survives the
row being posted back; a new row gets no value (the plugin fills it in).
- **The API is unaffected.** A JSON save to `POST /api/v3/plugins/config` can
still set a hidden field.
Older cores ignore the flag and render the field as a normal control.
## Creating Custom Widgets
### Step 1: Create Widget File
Create a JavaScript file in your plugin's `widgets/` directory, named
`widgets/[widget-name].js`. The directory is not optional: it is the only
place the core will serve a widget from.
```javascript
// Ensure LEDMatrixWidgets registry is available
if (typeof window.LEDMatrixWidgets === 'undefined') {
console.error('LEDMatrixWidgets registry not found');
return;
}
// Register your widget
window.LEDMatrixWidgets.register('my-custom-widget', {
name: 'My Custom Widget',
version: '1.0.0',
/**
* Render the widget HTML
* @param {HTMLElement} container - Container element to render into
* @param {Object} config - Widget configuration from schema
* @param {*} value - Current value
* @param {Object} options - Additional options (fieldId, pluginId, etc.)
*/
render: function(container, config, value, options) {
const fieldId = options.fieldId || container.id;
// Always escape HTML to prevent XSS
const escapeHtml = (text) => {
const div = document.createElement('div');
div.textContent = text;
return div.innerHTML;
};
container.innerHTML = `
<div class="my-custom-widget">
<input type="text"
id="${fieldId}_input"
value="${escapeHtml(value || '')}"
class="w-full px-3 py-2 border border-gray-300 rounded">
</div>
`;
// Attach event listeners
const input = container.querySelector('input');
input.addEventListener('change', (e) => {
this.handlers.onChange(fieldId, e.target.value);
});
},
/**
* Get current value from widget
*/
getValue: function(fieldId) {
const input = document.querySelector(`#${fieldId}_input`);
return input ? input.value : null;
},
/**
* Set value programmatically
*/
setValue: function(fieldId, value) {
const input = document.querySelector(`#${fieldId}_input`);
if (input) {
input.value = value || '';
}
},
/**
* Event handlers
*/
handlers: {
onChange: function(fieldId, value) {
// Trigger form change event
const event = new CustomEvent('widget-change', {
detail: { fieldId, value },
bubbles: true
});
document.dispatchEvent(event);
}
}
});
```
### Step 2: Declare the Widget in `manifest.json`
The manifest is the allowlist. A widget is served only if the plugin declares
it, so shipping a file under `widgets/` does not by itself publish it:
```json
{
"widgets": [
{
"name": "my-custom-widget",
"script": "my-custom-widget.js",
"description": "What this widget is for"
}
]
}
```
`name` is what you use in `x-widget` and in the URL. `script` is optional and
defaults to `[name].js`; it must be a plain filename directly inside
`widgets/` (no paths). Both are validated against
`schema/manifest_schema.json`.
### Step 3: Reference Widget in Schema
In your plugin's `config_schema.json`:
```json
{
"properties": {
"my_field": {
"type": "string",
"description": "My custom field",
"x-widget": "my-custom-widget",
"default": ""
}
}
}
```
### Step 4: Widget Loading
The widget is loaded on demand when the plugin's configuration form renders a
field that references it. The system will:
1. Check whether the widget is already registered in the core registry.
2. If not, fetch it from `/static/plugin-widgets/[plugin-id]/[widget-name].js`.
That route serves the declared `script` from your plugin's `widgets/`
directory, as `text/javascript`.
3. Render it by calling the `render` function your script registered.
The fetch uses a dynamic `import()`, so the file must parse as an ES module.
A plain IIFE does — modules are strict mode, so avoid sloppy-mode constructs.
**If the widget fails to load** (not declared, file missing, script throws, or
it never calls `register`), the field falls back to a plain text input holding
the current value. This is deliberate: a broken widget costs the user an
editor, not their configured value.
**Limitation:** the on-demand path applies to `string`-typed fields (the
default branch of the config-form renderer). Fields typed `object`, `array`,
`boolean`, `integer` or `number`, and fields whose `enum` is set, are
dispatched by the server-side template to its own built-in renderers, so a
plugin-supplied `x-widget` on one of those is ignored today.
## Widget API Reference
### Widget Definition Object
```javascript
{
name: string, // Human-readable widget name
version: string, // Widget version
render: function, // Required: Render function
getValue: function, // Optional: Get current value
setValue: function, // Optional: Set value programmatically
handlers: object // Optional: Event handlers
}
```
### Render Function
```javascript
render(container, config, value, options)
```
**Parameters:**
- `container` (HTMLElement): Container element to render into
- `config` (Object): Widget configuration from schema
- `value` (*): Current field value
- `options` (Object): Additional options
- `fieldId` (string): Field ID
- `pluginId` (string): Plugin ID
- `fullKey` (string): Full field key path
### Get Value Function
```javascript
getValue(fieldId)
```
**Returns:** Current widget value
### Set Value Function
```javascript
setValue(fieldId, value)
```
**Parameters:**
- `fieldId` (string): Field ID
- `value` (*): Value to set
## Examples
See [`web_interface/static/v3/js/widgets/example-color-picker.js`](../web_interface/static/v3/js/widgets/example-color-picker.js) for a complete example of a custom color picker widget.
## Best Practices
### Security
1. **Always escape HTML**: Use `escapeHtml()` or `textContent` to prevent XSS
2. **Validate inputs**: Validate user input before processing
3. **Sanitize values**: Clean values before storing
### Performance
1. **Lazy loading**: Load widget scripts only when needed
2. **Event delegation**: Use event delegation for dynamic content
3. **Debounce**: Debounce frequent events (e.g., input changes)
### Accessibility
1. **Labels**: Always associate labels with inputs
2. **ARIA attributes**: Use appropriate ARIA attributes
3. **Keyboard navigation**: Ensure keyboard accessibility
## Troubleshooting
### Widget Not Loading
1. Check browser console for errors
2. Verify widget file path is correct
3. Ensure `LEDMatrixWidgets.register()` is called
4. Check that widget name matches schema `x-widget` value
### Widget Not Rendering
1. Verify `render` function is defined
2. Check container element exists
3. Ensure widget is registered before form loads
4. Check for JavaScript errors in console
### Value Not Saving
1. Ensure widget triggers `widget-change` event
2. Verify form submission includes widget value
3. Check `getValue` function returns correct type
4. Verify field name matches schema property
## Current Implementation Status
**Phase 1 Complete:**
- ✅ Widget registry system created
- ✅ Core widgets extracted to separate files
- ✅ Widget handlers available globally (backwards compatible)
- ✅ Plugin widget loading system implemented
**Current Behavior:**
- Core widgets are server-side rendered via Jinja2 templates (existing behavior preserved)
- Widget handlers are registered and available globally
- Custom widgets can be created, declared in `manifest.json`, and are served
and rendered on demand for `string`-typed fields
- Plugin widgets on non-string fields are not dispatched yet (see Step 4)
**Backwards Compatibility:**
- All existing plugins using widgets continue to work without changes
- Server-side rendering remains the primary method
- Widget registry provides foundation for future enhancements
## See Also
- [Widget README](../web_interface/static/v3/js/widgets/README.md) - Complete widget development guide with examples
- [Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md) - General plugin development
- [Plugin Configuration Guide](PLUGIN_CONFIGURATION_GUIDE.md) - Configuration setup
+147 -108
View File
@@ -18,7 +18,7 @@ on_error() {
echo "-- Last 100 lines from log --" >&2
tail -n 100 "$LOG_FILE" >&2 || true
fi
printf '\nCommon fixes:\n' >&2
echo "\nCommon fixes:" >&2
echo "- Ensure the Pi is online (try: ping -c1 8.8.8.8)." >&2
echo "- If you saw an APT lock error: wait a minute, close other installers, then run: sudo dpkg --configure -a" >&2
echo "- Re-run this script. It is safe to run multiple times." >&2
@@ -115,8 +115,7 @@ fi
echo "✓ OS requirements met"
echo ""
# The user who ran the installer: SUDO_USER once we are running under sudo
# (the re-exec below guarantees that), otherwise whoever we are now.
# Get the actual user who invoked sudo (set after we ensure sudo below)
if [ -n "${SUDO_USER:-}" ]; then
ACTUAL_USER="$SUDO_USER"
else
@@ -203,7 +202,7 @@ echo ""
# Check if running as root; if not, try to elevate automatically for novices
if [ "$EUID" -ne 0 ]; then
echo "This script needs administrator privileges. Attempting to re-run with sudo..."
exec sudo -E bash "$0" "$@"
exec sudo -E env LEDMATRIX_ELEVATED=1 bash "$0" "$@"
fi
echo "✓ Running as root (required for installation)"
@@ -503,44 +502,6 @@ print_rgbmatrix_build_failure() {
fi
}
# Set WEB_SERVICE_USER to the account ledmatrix-web.service runs as, or "root"
# when it cannot tell. Steps 3.1 and 11 choose plugin-directory ownership from
# it. The logic was pasted three times, identically, and is kept verbatim here.
# Note: install_web_service.sh and install_service.sh no longer contain the
# "User=root" / "User=${ACTUAL_USER}" strings grepped for below (the units come
# from systemd/*.service templates with User=__USER__). So once the unit is
# installed (Step 7.5, by install_service.sh) the first branch reads its real
# User=; before that the second branch is taken whenever
# install_web_service.sh exists, matches neither string, and yields "root" --
# the later branches are reached only if that script is missing.
detect_web_service_user() {
WEB_SERVICE_USER="root"
if [ -f "/etc/systemd/system/ledmatrix-web.service" ]; then
# Check actual installed service file (most accurate)
WEB_SERVICE_USER=$(grep "^User=" /etc/systemd/system/ledmatrix-web.service | cut -d'=' -f2 || echo "root")
elif [ -f "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh" ]; then
# Check install_web_service.sh (used by first_time_install.sh)
if grep -q "User=root" "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh"; then
WEB_SERVICE_USER="root"
elif grep -q "User=\${ACTUAL_USER}" "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh"; then
WEB_SERVICE_USER="$ACTUAL_USER"
fi
elif [ -f "$PROJECT_ROOT_DIR/systemd/ledmatrix-web.service" ]; then
# Check template file (may have placeholder)
WEB_SERVICE_USER=$(grep "^User=" "$PROJECT_ROOT_DIR/systemd/ledmatrix-web.service" | cut -d'=' -f2 || echo "root")
# If template has placeholder, check install script
if [ "$WEB_SERVICE_USER" = "__USER__" ] || [ -z "$WEB_SERVICE_USER" ]; then
# Check install_service.sh to see what user it uses
if [ -f "$PROJECT_ROOT_DIR/scripts/install/install_service.sh" ] && grep -q "User=\${ACTUAL_USER}" "$PROJECT_ROOT_DIR/scripts/install/install_service.sh"; then
WEB_SERVICE_USER="$ACTUAL_USER"
fi
fi
elif [ -f "$PROJECT_ROOT_DIR/scripts/install/install_service.sh" ] && grep -q "User=\${ACTUAL_USER}" "$PROJECT_ROOT_DIR/scripts/install/install_service.sh"; then
# Web service will be installed by install_service.sh as ACTUAL_USER
WEB_SERVICE_USER="$ACTUAL_USER"
fi
}
echo ""
echo "This script will perform the following steps:"
echo "1. Check prerequisites (network, disk, memory) and install system dependencies"
@@ -673,9 +634,8 @@ else
echo "Setting ownership of assets directory..."
chown -R "$ACTUAL_USER:$ACTUAL_USER" "$PROJECT_ROOT_DIR/assets"
# 777: read/write for owner, group and every other account. Root (the
# display service) does not need it -- root ignores mode bits -- so the
# "other" bits only matter to accounts that are neither the owner nor root.
# Set permissions to allow read/write for owner, group, and others (for root service user)
# Note: 777 allows root (service user) to write, which is necessary when service runs as root
echo "Setting permissions for assets directory..."
chmod -R 777 "$PROJECT_ROOT_DIR/assets"
@@ -739,7 +699,32 @@ else
fi
# Determine ownership based on web service user
detect_web_service_user
# Check if web service file exists and what user it runs as
WEB_SERVICE_USER="root"
if [ -f "/etc/systemd/system/ledmatrix-web.service" ]; then
# Check actual installed service file (most accurate)
WEB_SERVICE_USER=$(grep "^User=" /etc/systemd/system/ledmatrix-web.service | cut -d'=' -f2 || echo "root")
elif [ -f "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh" ]; then
# Check install_web_service.sh (used by first_time_install.sh)
if grep -q "User=root" "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh"; then
WEB_SERVICE_USER="root"
elif grep -q "User=\${ACTUAL_USER}" "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh"; then
WEB_SERVICE_USER="$ACTUAL_USER"
fi
elif [ -f "$PROJECT_ROOT_DIR/systemd/ledmatrix-web.service" ]; then
# Check template file (may have placeholder)
WEB_SERVICE_USER=$(grep "^User=" "$PROJECT_ROOT_DIR/systemd/ledmatrix-web.service" | cut -d'=' -f2 || echo "root")
# If template has placeholder, check install script
if [ "$WEB_SERVICE_USER" = "__USER__" ] || [ -z "$WEB_SERVICE_USER" ]; then
# Check install_service.sh to see what user it uses
if [ -f "$PROJECT_ROOT_DIR/scripts/install/install_service.sh" ] && grep -q "User=\${ACTUAL_USER}" "$PROJECT_ROOT_DIR/scripts/install/install_service.sh"; then
WEB_SERVICE_USER="$ACTUAL_USER"
fi
fi
elif [ -f "$PROJECT_ROOT_DIR/scripts/install/install_service.sh" ] && grep -q "User=\${ACTUAL_USER}" "$PROJECT_ROOT_DIR/scripts/install/install_service.sh"; then
# Web service will be installed by install_service.sh as ACTUAL_USER
WEB_SERVICE_USER="$ACTUAL_USER"
fi
# If web service runs as ACTUAL_USER (not root), set ownership to ACTUAL_USER
# so the web service can change permissions. Root service can still access via group (775).
@@ -773,7 +758,32 @@ if [ ! -d "$PLUGIN_REPOS_DIR" ]; then
fi
# Determine ownership based on web service user
detect_web_service_user
# Check if web service file exists and what user it runs as
WEB_SERVICE_USER="root"
if [ -f "/etc/systemd/system/ledmatrix-web.service" ]; then
# Check actual installed service file (most accurate)
WEB_SERVICE_USER=$(grep "^User=" /etc/systemd/system/ledmatrix-web.service | cut -d'=' -f2 || echo "root")
elif [ -f "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh" ]; then
# Check install_web_service.sh (used by first_time_install.sh)
if grep -q "User=root" "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh"; then
WEB_SERVICE_USER="root"
elif grep -q "User=\${ACTUAL_USER}" "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh"; then
WEB_SERVICE_USER="$ACTUAL_USER"
fi
elif [ -f "$PROJECT_ROOT_DIR/systemd/ledmatrix-web.service" ]; then
# Check template file (may have placeholder)
WEB_SERVICE_USER=$(grep "^User=" "$PROJECT_ROOT_DIR/systemd/ledmatrix-web.service" | cut -d'=' -f2 || echo "root")
# If template has placeholder, check install script
if [ "$WEB_SERVICE_USER" = "__USER__" ] || [ -z "$WEB_SERVICE_USER" ]; then
# Check install_service.sh to see what user it uses
if [ -f "$PROJECT_ROOT_DIR/scripts/install/install_service.sh" ] && grep -q "User=\${ACTUAL_USER}" "$PROJECT_ROOT_DIR/scripts/install/install_service.sh"; then
WEB_SERVICE_USER="$ACTUAL_USER"
fi
fi
elif [ -f "$PROJECT_ROOT_DIR/scripts/install/install_service.sh" ] && grep -q "User=\${ACTUAL_USER}" "$PROJECT_ROOT_DIR/scripts/install/install_service.sh"; then
# Web service will be installed by install_service.sh as ACTUAL_USER
WEB_SERVICE_USER="$ACTUAL_USER"
fi
# If web service runs as ACTUAL_USER (not root), set ownership to ACTUAL_USER
# so the web service can change permissions. Root service can still access via group (775).
@@ -787,8 +797,8 @@ else
chown -R root:"$ACTUAL_USER" "$PLUGIN_REPOS_DIR"
fi
# Set directory permissions (2775: rwxrwsr-x, setgid so new entries inherit the group)
echo "Setting plugin-repos directory permissions to 2775 (setgid)..."
# Set directory permissions (775: rwxrwxr-x)
echo "Setting plugin-repos directory permissions to 2775 (sticky bit)..."
find "$PLUGIN_REPOS_DIR" -type d -exec chmod 2775 {} \;
# Set file permissions (664: rw-rw-r--)
@@ -995,9 +1005,9 @@ if [ -f "$PROJECT_ROOT_DIR/requirements.txt" ]; then
PACKAGE_NUM=$((PACKAGE_NUM + 1))
echo "[$PACKAGE_NUM/$TOTAL_PACKAGES] Installing: $line"
# Install with a timeout where available. --verbose output goes to
# $INSTALL_OUTPUT (filtered below, full copy in the log); --no-cache-dir
# avoids pip cache issues.
# Check if package is already installed (basic check - may not catch all cases)
# Try installing with verbose output and timeout (if available)
# Use --no-cache-dir to avoid cache issues, --verbose for diagnostics
INSTALL_OUTPUT=$(mktemp)
INSTALL_SUCCESS=false
@@ -1302,11 +1312,7 @@ else
WEB_DEPS_OK=false
fi
else
# No marker means Step 5 did not install web_interface/requirements.txt,
# and without the smart installer there is nothing else to try here.
echo "⚠ scripts/install_dependencies_apt.py not found, and Step 5 did not install"
echo " web_interface/requirements.txt, so web interface dependencies may be missing."
WEB_DEPS_OK=false
echo "Web dependencies already installed from web_interface/requirements.txt in Step 5"
fi
# Create the marker only when installation actually succeeded, so a
@@ -1510,31 +1516,51 @@ POWEROFF_PATH=$(which poweroff)
BASH_PATH=$(which bash)
JOURNALCTL_PATH=$(which journalctl 2>/dev/null || true)
# The rules themselves live in scripts/install/lib_sudoers.sh, shared with
# scripts/install/configure_web_sudo.sh so the two cannot drift apart again.
# If it is missing (a damaged checkout), keep whatever is already installed
# rather than failing the whole install; the gate below skips the install.
SUDOERS_VALID=1
SUDOERS_LIB="$PROJECT_ROOT_DIR/scripts/install/lib_sudoers.sh"
if [ -f "$SUDOERS_LIB" ]; then
# shellcheck source=scripts/install/lib_sudoers.sh
. "$SUDOERS_LIB"
web_sudoers_rules "$ACTUAL_USER" "$PROJECT_ROOT_DIR" "$SYSTEMCTL_PATH" "$BASH_PATH" \
"$REBOOT_PATH" "$POWEROFF_PATH" "$JOURNALCTL_PATH" > "$SUDOERS_TMP"
else
SUDOERS_VALID=0
echo "⚠ $SUDOERS_LIB not found; cannot generate the sudoers rules." >&2
echo "⚠ Leaving $SUDOERS_FILE unchanged. The web interface cannot control" >&2
echo " the display service until this is fixed." >&2
# Create sudoers content
cat > "$SUDOERS_TMP" << EOF
# LED Matrix Web Interface passwordless sudo configuration
# This allows the web interface user to run specific commands without a password
# Allow $ACTUAL_USER to run specific commands without a password for the LED Matrix web interface
$ACTUAL_USER ALL=(ALL) NOPASSWD: $REBOOT_PATH
$ACTUAL_USER ALL=(ALL) NOPASSWD: $POWEROFF_PATH
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH enable ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH disable ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH status ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix-web.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix-web.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/scripts/fix_perms/safe_plugin_rm.sh *
# Install a requirements.txt as root via vetted helper, so packages are visible
# to root-run ledmatrix.service (not just the web interface's own user).
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/scripts/fix_perms/safe_pip_install.sh *
EOF
if [ -n "$JOURNALCTL_PATH" ]; then
cat >> "$SUDOERS_TMP" << EOF
# NOEXEC, because these rules end in a wildcard and journalctl starts a pager
# when its output is a terminal. From that pager (less) a "!sh" is a root
# shell -- the standard journalctl escalation. The web interface always passes
# --no-pager, so nothing here needs it, but the rule cannot require a flag that
# sits in the middle of the command line. NOEXEC stops the command executing
# another program at all, which closes the hole without depending on wildcard
# matching subtleties.
$ACTUAL_USER ALL=(ALL) NOPASSWD:NOEXEC: $JOURNALCTL_PATH -u ledmatrix.service *
$ACTUAL_USER ALL=(ALL) NOPASSWD:NOEXEC: $JOURNALCTL_PATH -u ledmatrix *
$ACTUAL_USER ALL=(ALL) NOPASSWD:NOEXEC: $JOURNALCTL_PATH -t ledmatrix *
EOF
fi
# Never install rules we have not parsed. A malformed drop-in in
# /etc/sudoers.d makes sudo refuse every command for every user, which on a
# headless Pi leaves no way in at all. If the rules do not parse, say so and
# keep whatever is already installed.
if [ "$SUDOERS_VALID" = "0" ]; then
: # nothing was generated; already reported above
elif command -v visudo >/dev/null 2>&1; then
SUDOERS_VALID=1
if command -v visudo >/dev/null 2>&1; then
if ! visudo -c -f "$SUDOERS_TMP" >/dev/null 2>&1; then
SUDOERS_VALID=0
echo "⚠ The generated sudoers rules did not parse:" >&2
@@ -1571,12 +1597,11 @@ echo "-----------------------------------------------------"
if [ -f "$PROJECT_ROOT_DIR/scripts/install/configure_wifi_permissions.sh" ]; then
echo "Configuring WiFi management permissions..."
# Run as the actual user (not root) since the script checks for that
if sudo -u "$ACTUAL_USER" bash "$PROJECT_ROOT_DIR/scripts/install/configure_wifi_permissions.sh"; then
echo "✓ WiFi management permissions configured"
else
sudo -u "$ACTUAL_USER" bash "$PROJECT_ROOT_DIR/scripts/install/configure_wifi_permissions.sh" || {
echo "⚠ WiFi permissions configuration failed, but continuing installation"
echo " You can run it manually later: ./scripts/install/configure_wifi_permissions.sh"
fi
}
echo "✓ WiFi management permissions configured"
else
echo "⚠ configure_wifi_permissions.sh not found; skipping WiFi permissions configuration"
echo " You can configure WiFi permissions later by running:"
@@ -1665,8 +1690,28 @@ fi
# Re-apply plugin directory permissions based on web service user
echo "Re-applying plugin directory permissions..."
# Determine ownership based on web service user
detect_web_service_user
# Determine web service user (check installed service, install scripts, or template)
WEB_SERVICE_USER="root"
if [ -f "/etc/systemd/system/ledmatrix-web.service" ]; then
# Check actual installed service file (most accurate)
WEB_SERVICE_USER=$(grep "^User=" /etc/systemd/system/ledmatrix-web.service | cut -d'=' -f2 || echo "root")
elif [ -f "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh" ]; then
# Check install_web_service.sh (used by first_time_install.sh)
if grep -q "User=root" "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh"; then
WEB_SERVICE_USER="root"
elif grep -q "User=\${ACTUAL_USER}" "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh"; then
WEB_SERVICE_USER="$ACTUAL_USER"
fi
elif [ -f "$PROJECT_ROOT_DIR/systemd/ledmatrix-web.service" ]; then
WEB_SERVICE_USER=$(grep "^User=" "$PROJECT_ROOT_DIR/systemd/ledmatrix-web.service" | cut -d'=' -f2 || echo "root")
if [ "$WEB_SERVICE_USER" = "__USER__" ] || [ -z "$WEB_SERVICE_USER" ]; then
if [ -f "$PROJECT_ROOT_DIR/scripts/install/install_service.sh" ] && grep -q "User=\${ACTUAL_USER}" "$PROJECT_ROOT_DIR/scripts/install/install_service.sh"; then
WEB_SERVICE_USER="$ACTUAL_USER"
fi
fi
elif [ -f "$PROJECT_ROOT_DIR/scripts/install/install_service.sh" ] && grep -q "User=\${ACTUAL_USER}" "$PROJECT_ROOT_DIR/scripts/install/install_service.sh"; then
WEB_SERVICE_USER="$ACTUAL_USER"
fi
# Set ownership based on web service user
if [ "$WEB_SERVICE_USER" = "$ACTUAL_USER" ] || [ "$WEB_SERVICE_USER" != "root" ]; then
@@ -1746,11 +1791,10 @@ echo "-------------------------------------"
echo "Removing potential conflicting services (bluetooth and others)..."
if [ "$SKIP_SOUND" = "1" ]; then
echo "Skipping sound module configuration as requested (--skip-sound)."
else
# apt_remove never fails (it ends in `|| true`); apt itself reports any
# package it could not remove.
apt_remove bluez bluez-firmware pi-bluetooth triggerhappy pigpio
elif apt_remove bluez bluez-firmware pi-bluetooth triggerhappy pigpio; then
echo "✓ Unnecessary services removed (or not present)"
else
echo "⚠ Some packages could not be removed; continuing"
fi
# Blacklist onboard sound module (idempotent)
@@ -1965,6 +2009,20 @@ if systemctl list-unit-files | grep -q "ledmatrix-wifi-monitor.service"; then
fi
echo ""
if [ "$SKIP_REBOOT_PROMPT" = "1" ]; then
echo "Skipping reboot prompt as requested (--no-reboot-prompt)."
elif [ "$ASSUME_YES" = "1" ]; then
echo "Non-interactive mode: rebooting now to apply changes..."
reboot
else
read -p "A reboot is recommended to apply kernel and audio changes. Reboot now? (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
echo "Rebooting now..."
reboot
fi
fi
echo "=========================================="
echo "Installation Complete!"
echo "=========================================="
@@ -2010,7 +2068,7 @@ if command -v nmcli >/dev/null 2>&1; then
if [ -n "$WIFI_STATUS" ]; then
echo "$WIFI_STATUS" | while IFS=':' read -r _ _ state; do
if [ "$state" = "connected" ]; then
SSID=$(nmcli -t -f active,ssid device wifi 2>/dev/null | grep "^yes:" | cut -d: -f2 | head -1 || true)
SSID=$(nmcli -t -f active,ssid device wifi 2>/dev/null | grep "^yes:" | cut -d: -f2 | head -1)
if [ -n "$SSID" ]; then
echo " ✓ Connected to: $SSID"
else
@@ -2034,7 +2092,7 @@ echo "AP Mode Status:"
if systemctl is-active --quiet hostapd 2>/dev/null; then
echo " ✓ AP Mode is ACTIVE"
echo " → Connect to WiFi network: LEDMatrix-Setup"
echo " → Open network, no password"
echo " → Password: ledmatrix123"
echo " → Access web UI at: http://192.168.4.1:5000"
AP_MODE_ACTIVE=true
else
@@ -2042,7 +2100,7 @@ else
if ip addr show wlan0 2>/dev/null | grep -q "192.168.4.1"; then
echo " ✓ AP Mode is ACTIVE (IP detected)"
echo " → Connect to WiFi network: LEDMatrix-Setup"
echo " → Open network, no password"
echo " → Password: ledmatrix123"
echo " → Access web UI at: http://192.168.4.1:5000"
AP_MODE_ACTIVE=true
else
@@ -2144,22 +2202,3 @@ echo " - Main config: $PROJECT_ROOT_DIR/config/config.json"
echo " - Secrets: $PROJECT_ROOT_DIR/config/config_secrets.json"
echo ""
echo "Enjoy your LED Matrix display!"
# Reboot last. It used to come before the summary above, so with -y (and
# the one-shot installer, which always passes -y) the reboot was already
# under way while the summary printed, and the SSH session usually dropped
# before any of it -- the web UI address included -- could be read.
echo ""
if [ "$SKIP_REBOOT_PROMPT" = "1" ]; then
echo "Skipping reboot prompt as requested (--no-reboot-prompt)."
elif [ "$ASSUME_YES" = "1" ]; then
echo "Non-interactive mode: rebooting now to apply changes..."
reboot
else
read -p "A reboot is recommended to apply kernel and audio changes. Reboot now? (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
echo "Rebooting now..."
reboot
fi
fi
-83
View File
@@ -1,83 +0,0 @@
# The mypy ratchet: modules that type-check clean, one path per line, sorted.
#
# CI ("Type check (mypy ratchet)") runs `python scripts/check_types.py`, which
# runs mypy on exactly these files (imports followed silently, so errors in an
# unlisted module they import don't count) and fails on any error, so a listed
# module stays clean. Most of src/ isn't clean yet. When you make a module
# clean, add it here. Don't take a module off to get CI green -- fix the error
# (annotation-only where you can: hints, typing.cast, TYPE_CHECKING imports;
# widen an annotation rather than delete a defensive runtime check).
src/__init__.py
src/adaptive_images.py
src/auto_update_setup.py
src/backup_manager.py
src/base_odds_manager.py
src/cache/__init__.py
src/cache/cache_metrics.py
src/cache/cache_strategy.py
src/cache/memory_cache.py
src/common/__init__.py
src/common/api_helper.py
src/common/bdf_font.py
src/common/espn_dates.py
src/common/favorite_team_check.py
src/common/font_layout.py
src/common/frame_timing.py
src/common/json_body.py
src/common/logo_helper.py
src/common/path_safety.py
src/common/permission_utils.py
src/common/render_gate.py
src/common/scroll_config.py
src/common/snapshot_policy.py
src/common/sports_card.py
src/common/sports_scroll.py
src/common/sports_timezone.py
src/config_service.py
src/core_config_keys.py
src/deprecation.py
src/device_location.py
src/display_geometry.py
src/dynamic_team_resolver.py
src/exceptions.py
src/font_usage.py
src/logging_config.py
src/logo_downloader.py
src/matrix_support.py
src/pi5_matrix_support.py
src/plugin_system/__init__.py
src/plugin_system/compatibility.py
src/plugin_system/operation_history.py
src/plugin_system/operation_queue.py
src/plugin_system/operation_types.py
src/plugin_system/plugin_dirs.py
src/plugin_system/plugin_executor.py
src/plugin_system/plugin_health.py
src/plugin_system/plugin_loader.py
src/plugin_system/plugin_state.py
src/plugin_system/repo_urls.py
src/plugin_system/resource_monitor.py
src/plugin_system/saved_repositories.py
src/plugin_system/schema_manager.py
src/plugin_system/state_reconciliation.py
src/plugin_system/testing/__init__.py
src/plugin_system/testing/bounds_display_manager.py
src/plugin_system/testing/loading.py
src/plugin_system/testing/mocks.py
src/plugin_system/testing/plugin_test_base.py
src/plugin_system/testing/sizes.py
src/redaction.py
src/scan_order.py
src/startup_validator.py
src/vegas_mode/__init__.py
src/vegas_mode/config.py
src/vegas_mode/coordinator.py
src/vegas_mode/geometry.py
src/vegas_mode/stream_manager.py
src/web_interface/api_helpers.py
src/web_interface/config_arrays.py
src/web_interface/error_handler.py
src/web_interface/errors.py
src/web_interface/secret_helpers.py
src/web_interface/validators.py
+10 -25
View File
@@ -1,11 +1,6 @@
[mypy]
# Mypy configuration for LEDMatrix
# What a bare `mypy` checks. ini values can't span lines or carry trailing
# comments -- this file used to have both, so mypy refused to read it at all.
files = src
exclude = (^|/)(test|__pycache__)/
# Python version
python_version = 3.10
@@ -30,11 +25,11 @@ warn_unreachable = True
# Strict optional checking
strict_optional = True
# Disallow untyped definitions (set to True once all code is typed)
disallow_untyped_defs = False
# Disallow untyped definitions
disallow_untyped_defs = False # Set to True once all code is typed
# Disallow untyped calls (set to True once all code is typed)
disallow_untyped_calls = False
# Disallow untyped calls
disallow_untyped_calls = False # Set to True once all code is typed
# Check untyped definitions
check_untyped_defs = True
@@ -101,20 +96,10 @@ ignore_missing_imports = True
[mypy-spotipy.*]
ignore_missing_imports = True
# Exclude test files and generated files
exclude = (?x)(
^test/.*|
^.*/__pycache__/.*|
^.*\.pyc$
)
# numpy's own stubs (numpy>=2.3) use Python 3.12 `type` statements, which mypy
# refuses to parse under python_version = 3.10 -- and 3.10 is the floor this
# code has to run on, so it stays. Treat numpy as Any instead: skip it, and
# follow_imports_for_stubs makes the skip apply to its .pyi files too.
[mypy-numpy.*]
follow_imports = skip
follow_imports_for_stubs = True
# orjson is optional (see requirements.txt): the modules that use it fall back
# to the stdlib when `import orjson` fails. Whether mypy sees its stubs would
# otherwise depend on whether it happens to be installed -- installed, the
# `orjson = None` fallback is a type error and the stdlib branch "unreachable";
# not installed, silencing either is an unused ignore. Treat it as Any always.
[mypy-orjson.*]
follow_imports = skip
follow_imports_for_stubs = True
+1 -83
View File
@@ -10,7 +10,6 @@ API Version: 1.0.0
import json
import os
import re
import stat
import time
import fcntl
from pathlib import Path
@@ -19,7 +18,6 @@ from PIL import Image
from src.plugin_system.base_plugin import BasePlugin, VegasDisplayMode
from src.logging_config import get_logger
from src.device_location import DeviceLocationResolver, apply_device_location
from pixlet_renderer import PixletRenderer
from frame_extractor import FrameExtractor
@@ -230,10 +228,6 @@ class StarlarkAppsPlugin(BasePlugin):
self.current_app: Optional[StarlarkApp] = None
self.last_update_check = 0
# Unset location fields render at the device's location, not the
# app author's default (usually San Francisco).
self.device_location = DeviceLocationResolver(cache_manager, self.logger)
# Check Pixlet availability
if not self.pixlet.is_available():
self.logger.error("Pixlet not available - Starlark apps will not work")
@@ -463,77 +457,12 @@ class StarlarkAppsPlugin(BasePlugin):
apps_dir = project_root / "starlark-apps"
except Exception:
# Fallback to current working directory
project_root = Path.cwd()
apps_dir = project_root / "starlark-apps"
apps_dir = Path.cwd() / "starlark-apps"
# Create directory if it doesn't exist
apps_dir.mkdir(parents=True, exist_ok=True)
self._hand_apps_dir_to_checkout_owner(apps_dir, project_root)
return apps_dir
def _hand_apps_dir_to_checkout_owner(self, apps_dir: Path, project_root: Path) -> None:
"""Give the apps directory to whoever owns the checkout.
This directory is not in the repository, so it is created lazily by
whichever process reaches it first -- and the two that do run as
different users. The display service is `User=root`
(systemd/ledmatrix.service) and instantiates this plugin at startup,
which is where `_get_apps_directory` is called from. The web interface
is `User=<login user>` (systemd/ledmatrix-web.service) and is what
actually installs apps.
On a fresh install the display service usually wins that race -- the
documented first step is to install pixlet and reboot -- so the
directory lands root-owned, and every subsequent install from the web
UI fails on PermissionError. The user sees only "Failed to install
from repository", with nothing pointing at ownership.
The web user cannot repair this; it lacks permission to chown. Root
can, so root does it here, on every startup. That also heals installs
already broken by this, without the user having to find the chown.
"""
geteuid = getattr(os, "geteuid", None)
chown = getattr(os, "chown", None)
if geteuid is None or chown is None or geteuid() != 0:
# Not root, or not a platform with POSIX ownership. If the
# directory is wrong we cannot fix it, and must not pretend to.
return
try:
owner = project_root.stat()
except OSError:
return
if owner.st_uid == 0:
# The checkout genuinely belongs to root, so root owning the apps
# directory is correct and there is nobody to hand it to.
return
# Deepest first, with the directory itself last. Handing over the
# container before its contents would briefly let a local user rename
# entries underneath a repair that is still running.
descendants = sorted(apps_dir.rglob("*"),
key=lambda p: len(p.parts), reverse=True)
for path in (*descendants, apps_dir):
try:
st = os.lstat(path)
except OSError:
continue
if stat.S_ISLNK(st.st_mode):
# Never hand over a link's target. Anyone able to write in
# this directory could otherwise point a symlink at a
# root-owned file and have this give it away -- the whole
# point of the loop is that it runs as root.
continue
if st.st_uid == owner.st_uid and st.st_gid == owner.st_gid:
continue
try:
chown(path, owner.st_uid, owner.st_gid, follow_symlinks=False)
except OSError as e:
self.logger.warning(
"Could not hand %s to uid %s: %s -- installs from the web "
"interface will fail until this is chowned manually",
path, owner.st_uid, e,
)
def _sanitize_app_id(self, app_id: str) -> str:
"""
Sanitize app_id into a safe slug for use in file paths.
@@ -887,10 +816,6 @@ class StarlarkAppsPlugin(BasePlugin):
# Filter out LEDMatrix-internal timing/sizing keys before passing to pixlet
INTERNAL_KEYS = {'render_interval', 'display_duration', 'render_width', 'render_height'}
pixlet_config = {k: v for k, v in app.config.items() if k not in INTERNAL_KEYS}
# Applied here rather than saved into config.json, so a later
# change to the device location reaches the next render.
pixlet_config = apply_device_location(
pixlet_config, app.schema, self.device_location, self.global_config)
success, error = self.pixlet.render(
star_file=str(app.star_file),
@@ -1086,13 +1011,6 @@ class StarlarkAppsPlugin(BasePlugin):
self.logger.info(f"Installed Starlark app: {app_id} (sanitized: {safe_app_id})")
return True
except PermissionError:
# Deliberately not folded into the False below. A False here is
# reported as a generic install failure, which is how the
# directory-ownership bug stayed invisible: the caller could not
# tell "this app is broken" from "this process cannot write here".
# The routes turn this into a message that names the fix.
raise
except Exception as e:
self.logger.error(f"Error installing app {app_id}: {e}")
return False
+2 -2
View File
@@ -1,10 +1,10 @@
# Test/dev-only dependencies (not needed on a running display).
# Install alongside requirements.txt: pip install -r requirements.txt -r requirements-test.txt
pytest>=9.0.3,<10.0.0
pytest-cov>=4.1.0,<8.0.0
pytest-cov>=4.1.0,<5.0.0
pytest-mock>=3.11.0,<4.0.0
freezegun>=1.2,<2 # deterministic time for golden-image tests
psutil>=6.0.0,<7.0.0 # optional at runtime; installed for tests so the
psutil>=6.0.0,<8.0.0 # optional at runtime; installed for tests so the
# /system/status endpoint's real path is exercised
mypy>=1.5.0,<2.0.0 # static type checking (also pinned in .pre-commit-config.yaml)
PyYAML>=6.0.2,<7.0.0 # not a core dependency: test_starlark_pixlet_routes loads
+1 -1
View File
@@ -7,7 +7,7 @@ Pillow>=12.2.0,<13.0.0
numpy>=1.24.0 # For fast array operations in ScrollHelper (compatible with 2.x)
# Timezone handling
pytz>=2024.2,<2027.0 # Updated for latest timezone data
pytz>=2024.2,<2025.0 # Updated for latest timezone data
# HTTP requests
requests>=2.33.0,<3.0.0
+2
View File
@@ -31,6 +31,8 @@ if args.emulator:
print("Using pygame/RGBMatrixEmulator for display")
print("Press ESC to exit\n")
# Project directory already added above
# Debug output (only in debug mode or emulator mode)
debug_mode = args.debug or args.emulator or os.environ.get('LEDMATRIX_DEBUG', '').lower() == 'true'
if debug_mode:
-61
View File
@@ -1,61 +0,0 @@
# Scripts
Helper scripts for installing, repairing, diagnosing and developing
LEDMatrix. Most users only ever run the one-shot installer (see the project
README); everything else here is for troubleshooting or development.
Status key: **keep** — part of install/runtime or referenced by docs, CI,
tests or code; **dev-only** — for plugin/core development, not needed on a
display; **diagnostic** — run by hand on a Pi when something is wrong.
## Directories
| Directory | Status | What it holds |
|---|---|---|
| [`install/`](install/README.md) | keep | The installers: one-shot, services, sudoers/WiFi permissions, cache setup, and the shared `lib_*.sh` helpers `first_time_install.sh` sources |
| [`fix_perms/`](fix_perms/README.md) | keep | Permission repair scripts, plus the two root helpers the web interface runs through sudo (`safe_plugin_rm.sh`, `safe_pip_install.sh`) |
| [`utils/`](utils/README.md) | keep | Scripts run by systemd units or the web interface (conditional web start, WiFi monitor, update verify, DNS fix, Pixlet config editor, cache clearing) |
| [`dev/`](dev/README.md) | dev-only | Plugin linking, emulator runner, Vegas density audit, Pillow smoke test |
| `templates/` | dev-only | `dev_preview.html`, the page `dev_server.py` serves |
## Top-level scripts
| Script | Status | What it does |
|---|---|---|
| `build_rgbmatrix_nogil.sh` | keep | Rebuilds the rgbmatrix Python binding so `SwapOnVSync` releases the GIL (docs/SCROLL_PERFORMANCE.md) |
| `check_plugin.py` | dev-only | Renders a plugin across every mode and matrix size and fails on crashes, overflow or golden-image drift |
| `check_release_version.py` | keep | Checks a release tag, CHANGELOG and `src.__version__` agree (release-version-check workflow) |
| `check_system_compatibility.sh` | diagnostic | Pre-install check of hardware, OS (Trixie only), kernel, Python, packages, disk and network |
| `dev_server.py` | dev-only | Browser preview server for plugins without the display loop (http://localhost:5001) |
| `diagnose_dependencies.sh` | diagnostic | Investigates pip installs stuck on "Preparing metadata" |
| `diagnose_web_interface.sh` | diagnostic | Checks why the web interface is not reachable |
| `download_pixlet.sh` | keep | Downloads the bundled Pixlet binaries for Starlark apps (also run from the web UI) |
| `emergency_reconnect.sh` | diagnostic | Reconnects to your WiFi network if captive-portal testing leaves the Pi offline |
| `frame_soak.py` | diagnostic | Soaks a running display and reports how often frames reached the panel late (docs/SCROLL_PERFORMANCE.md) |
| `install_dependencies_apt.py` | keep | Dependency installer that tries apt packages first, then pip (installer Step 7, plugin loader) |
| `install_plugin_dependencies.sh` | diagnostic | Installs plugin requirements by hand when the automatic install fails |
| `prove_security.py` | keep | Security property checks run by pre-commit |
| `render_bench.py` | diagnostic | Benchmarks the render loop against the panel's real refresh rate on a synthetic strip |
| `render_plugin.py` | dev-only | Runs a plugin's `update()` + `display()` and saves the frame as a PNG |
| `run_plugin_tests.py` | dev-only | Discovers and runs plugin test suites |
| `scroll_speeds.py` | keep | Shows and tries the scroll speeds your panel can display cleanly |
| `troubleshoot_captive_portal.sh` | diagnostic | Troubleshoots captive-portal WiFi setup after you can SSH back in |
| `update_plugin_repos.py` | dev-only | Pulls the latest `ledmatrix-plugins` monorepo |
| `verify_installation.sh` | diagnostic | Checks that an installation completed correctly |
| `verify_wifi_setup.sh` | diagnostic | Health check of the WiFi management setup |
## Candidates for removal
Nothing in the repo (docs, CI, tests, other scripts or code) refers to these.
They are kept for now; each one needs an owner decision before it goes.
| Script | What it does |
|---|---|
| `add_defaults_to_schemas.py` | One-off: adds missing `default` values to plugin config schemas |
| `analyze_plugin_schemas.py` | One-off: reports duplicate/inconsistent fields across plugin schemas |
| `audit_plugins.py` | AST security audit of plugin code; says it is "designed to run in CI" but no workflow runs it |
| `audit_render_path.py` | Finds blocking calls reachable from a plugin's `display()` |
| `sports_scroll_check.py` | Drives a sports scoreboard scroll on the panel and reports its pacing |
| `test_captive_portal.sh` | Tests the captive portal from a device connected to the AP |
| `verify_wifi_before_testing.sh` | Pre-flight check before unplugging Ethernet to test WiFi |
| `dev/test_pillow_compat.py` | Pillow API smoke test to run after upgrading Pillow |
+1 -1
View File
@@ -194,7 +194,7 @@ def process_schema_file(schema_path: Path) -> bool:
print(f" ✓ Modified {len(modified_fields)} fields")
return True
else:
print(" ✓ No changes needed")
print(f" ✓ No changes needed")
return False
+3 -2
View File
@@ -87,6 +87,7 @@ def find_duplicate_fields(schema: Dict[str, Any], path: str = "") -> List[str]:
def validate_schema_syntax(schema_path: Path) -> tuple[bool, List[str]]:
"""Validate JSON Schema syntax."""
errors = []
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
@@ -163,7 +164,7 @@ def analyze_schema(schema_path: Path) -> Dict[str, Any]:
if "update_interval_seconds" in properties:
analysis["update_interval_variant"] = "update_interval_seconds"
analysis["naming_issues"].append(
"Uses 'update_interval_seconds' instead of 'update_interval'"
f"Uses 'update_interval_seconds' instead of 'update_interval'"
)
else:
analysis["missing_common_fields"].append(field_name)
@@ -238,7 +239,7 @@ def main():
print(f" Missing common fields: {', '.join(result['missing_common_fields'])}")
if result['naming_issues']:
print(" Naming issues:")
print(f" Naming issues:")
for issue in result['naming_issues']:
print(f" - {issue}")
+19 -18
View File
@@ -28,12 +28,12 @@ print_success() {
print_warning() {
echo -e "${YELLOW}⚠${NC} $1"
WARNINGS=$((WARNINGS + 1))
((WARNINGS++))
}
print_error() {
echo -e "${RED}✗${NC} $1"
COMPATIBILITY_ISSUES=$((COMPATIBILITY_ISSUES + 1))
((COMPATIBILITY_ISSUES++))
}
# Check if running on Raspberry Pi
@@ -61,18 +61,20 @@ if [ -f /etc/os-release ]; then
echo "OS: $PRETTY_NAME"
echo "Version ID: ${VERSION_ID:-unknown}"
# first_time_install.sh refuses anything but Raspberry Pi OS / Debian 13
# (Trixie), so anything else is an error here too, not a warning.
if [[ "$ID" == "raspbian" ]] || [[ "$ID" == "debian" ]]; then
if [ "${VERSION_ID:-0}" = "13" ]; then
print_success "Detected Debian 13 Trixie - supported"
elif [ "${VERSION_ID:-0}" = "12" ]; then
print_error "Debian 12 Bookworm is not supported - the installer requires Raspberry Pi OS Lite (Trixie), Debian 13"
if [ "${VERSION_ID:-0}" -ge "12" ]; then
print_success "Running compatible Debian/Raspbian version (${VERSION_ID})"
if [ "${VERSION_ID:-0}" -eq "13" ]; then
print_success "Detected Debian 13 Trixie - full compatibility expected"
elif [ "${VERSION_ID:-0}" -eq "12" ]; then
print_success "Detected Debian 12 Bookworm - full compatibility confirmed"
fi
else
print_error "Debian/Raspbian ${VERSION_ID:-unknown} is not supported - the installer requires Raspberry Pi OS Lite (Trixie), Debian 13"
print_warning "Old Debian/Raspbian version (${VERSION_ID}) - upgrade recommended"
fi
else
print_error "${ID:-unknown} is not supported - the installer requires Raspberry Pi OS Lite (Trixie), Debian 13"
print_warning "Not running Debian/Raspbian - compatibility not guaranteed"
fi
else
print_error "Could not detect OS version"
@@ -112,13 +114,15 @@ if command -v python3 >/dev/null 2>&1; then
echo "Python: $PYTHON_VERSION"
if [ "$PYTHON_MAJOR" -eq "3" ]; then
if [ "$PYTHON_MINOR" -ge "10" ] && [ "$PYTHON_MINOR" -le "13" ]; then
print_success "Python version is supported (3.10-3.13)"
if [ "$PYTHON_MINOR" -ge "10" ] && [ "$PYTHON_MINOR" -le "12" ]; then
print_success "Python version is fully supported (3.10-3.12)"
elif [ "$PYTHON_MINOR" -eq "13" ]; then
print_warning "Python 3.13 detected - most packages compatible, but some may have limited testing"
print_warning "Please report any compatibility issues you encounter"
elif [ "$PYTHON_MINOR" -ge "14" ]; then
print_warning "Python 3.${PYTHON_MINOR} is very new - some packages may not be compatible yet"
else
# Pillow 12 and the pinned test tools need 3.10+, so this won't install.
print_error "Python 3.${PYTHON_MINOR} is too old - Python 3.10+ is required"
print_warning "Python 3.${PYTHON_MINOR} is outdated - upgrade to 3.10+ recommended"
fi
else
print_error "Python 2.x detected - Python 3.10+ is required"
@@ -153,10 +157,7 @@ ESSENTIAL_PACKAGES=(
for pkg_info in "${ESSENTIAL_PACKAGES[@]}"; do
IFS=':' read -r pkg desc <<< "$pkg_info"
# dpkg-query rather than `dpkg -l | grep -q`: under pipefail, grep -q
# exiting on its first match kills dpkg with SIGPIPE and fails the pipeline,
# which reported installed packages as missing.
if [ "$(dpkg-query -W -f='${Status}' "$pkg" 2>/dev/null)" = "install ok installed" ]; then
if dpkg -l | grep -q "^ii $pkg "; then
print_success "$desc ($pkg) is installed"
else
print_warning "$desc ($pkg) not installed - will be installed during setup"
-99
View File
@@ -1,99 +0,0 @@
#!/usr/bin/env python3
"""Type-check the modules listed in mypy-clean.txt (the mypy ratchet).
Most of src/ still has mypy errors, so CI can't require a clean `mypy src`.
Instead mypy-clean.txt lists the modules that *are* clean, and this script
fails if any of them regresses. When you make another module clean, add it to
the list; nothing ever comes off it.
Imports are followed silently: a listed module is checked against the types of
everything it imports, but errors inside those imported modules are not
reported, so a clean file isn't failed by an unlisted neighbour.
Usage:
python scripts/check_types.py # check the listed modules
python scripts/check_types.py --list # print the list and exit
Extra arguments after ``--`` are passed to mypy.
Exit status: 0 clean, 1 mypy errors, 2 a bad list (missing file, duplicate,
unsorted, or empty).
"""
import argparse
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
LIST_FILE = REPO_ROOT / "mypy-clean.txt"
def read_list(path: Path = LIST_FILE) -> list:
"""The listed paths, in file order, with comments and blank lines dropped."""
entries = []
for raw in path.read_text(encoding="utf-8").splitlines():
line = raw.split("#", 1)[0].strip()
if line:
entries.append(line)
return entries
def list_problems(entries: list, root: Path = REPO_ROOT) -> list:
"""Why the list can't be used as-is; empty when it is fine."""
problems = []
if not entries:
problems.append(f"{LIST_FILE.name} lists no modules")
seen = set()
for entry in entries:
if entry in seen:
problems.append(f"listed twice: {entry}")
seen.add(entry)
if "\\" in entry:
problems.append(f"use forward slashes: {entry}")
elif not (root / entry).is_file():
problems.append(f"listed but not found (renamed or deleted? update the list): {entry}")
if entries != sorted(entries):
problems.append(f"{LIST_FILE.name} is not sorted")
return problems
def main(argv=None) -> int:
parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
parser.add_argument("--list", action="store_true", help="print the listed modules and exit")
parser.add_argument("mypy_args", nargs="*", help="extra mypy arguments (after --)")
args = parser.parse_args(argv)
entries = read_list()
problems = list_problems(entries)
if problems:
for problem in problems:
print(f"check_types: {problem}", file=sys.stderr)
return 2
if args.list:
print("\n".join(entries))
return 0
cmd = [
sys.executable, "-m", "mypy",
"--config-file", str(REPO_ROOT / "mypy.ini"),
"--follow-imports=silent",
*args.mypy_args,
*entries,
]
print(f"check_types: mypy on {len(entries)} modules from {LIST_FILE.name}", flush=True)
# This interpreter's mypy, fixed flags, and paths from the checked-in list.
result = subprocess.run(cmd, cwd=REPO_ROOT) # nosec B603 - list-form argv, no shell # nosemgrep
if result.returncode > 1: # mypy itself failed (bad config, crash)
return result.returncode
if result.returncode != 0:
print(
"check_types: a module on the mypy ratchet has type errors. Fix them "
f"(annotation-only where possible) rather than taking it off {LIST_FILE.name}.",
file=sys.stderr,
)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
-2
View File
@@ -6,8 +6,6 @@ This directory contains scripts and utilities for development and testing.
- **`dev_plugin_setup.sh`** - Sets up plugin development environment by linking plugin repositories
- **`run_emulator.sh`** - Runs the LED Matrix display in emulator mode (for development without hardware)
- **`vegas_audit.py`** - Measures how much of the Vegas ticker strip actually shows content (dead-frame ratio)
- **`test_pillow_compat.py`** - Pillow API smoke test to run after upgrading Pillow (`python3 scripts/dev/test_pillow_compat.py`)
## Usage
+1 -1
View File
@@ -423,7 +423,7 @@ def main():
global _extra_dirs
_extra_dirs = args.extra_dir
print("LEDMatrix Dev Preview Server")
print(f"LEDMatrix Dev Preview Server")
print(f"Open http://{args.host}:{args.port} in your browser")
print(f"Plugin search dirs: {[str(d) for d in get_search_dirs()]}")
print()
+10 -17
View File
@@ -18,26 +18,21 @@ system user.
permissions on the `assets/` tree so plugins can download and cache
team logos, fonts, and other static content.
- **`fix_cache_permissions.sh`** — Restores `/var/cache/ledmatrix/` to the
shared `ledmatrix`-group setup by running
`scripts/install/setup_cache.sh` (the same script the installer uses),
and creates/fixes `~/.ledmatrix_cache/` of the user running `sudo`. It
does not touch the cache manager's other fallbacks
(`/opt/ledmatrix/cache`, `$TMPDIR/ledmatrix_cache`).
- **`fix_cache_permissions.sh`** — Creates (if missing) and fixes
permissions on `/var/cache/ledmatrix/` and `~/.ledmatrix_cache/` of the
user running `sudo`, and creates
`/var/cache/ledmatrix/placeholder_logos/` for the sports plugins. It does
not touch the cache manager's other fallbacks (`/opt/ledmatrix/cache`,
`$TMPDIR/ledmatrix_cache`).
- **`fix_plugin_permissions.sh`** — Fixes ownership on the plugins
directory so both the root display service and the web service user
can read and write plugin files (manifests, configs, requirements
installs).
- **`fix_web_permissions.sh`** — Adds you to the `systemd-journal` and
`adm` groups so the web UI can read logs, and makes the project
directory yours again, keeping the root-owned sudo helpers
(`safe_plugin_rm.sh`, `safe_pip_install.sh`) and `config_secrets.json`
the way the installer leaves them. Run it as the web interface's user,
**without** `sudo` (it refuses to run as root and calls `sudo` itself).
It does not write sudoers rules; that is
`scripts/install/configure_web_sudo.sh`.
- **`fix_web_permissions.sh`** — Fixes permissions on log files,
systemd journal access, and the sudoers entries the web interface
needs to control the display service.
- **`safe_pip_install.sh`** — Installs a `requirements.txt` as root
after checking it is the project's own or one under `plugin-repos/` or
@@ -72,9 +67,7 @@ Run these scripts only when:
sudo ./scripts/fix_perms/fix_cache_permissions.sh
sudo ./scripts/fix_perms/fix_assets_permissions.sh
sudo ./scripts/fix_perms/fix_plugin_permissions.sh
# Run as the web interface's user, without sudo (it asks for sudo itself)
./scripts/fix_perms/fix_web_permissions.sh
sudo ./scripts/fix_perms/fix_web_permissions.sh
```
If you're not sure which one you need, run `fix_cache_permissions.sh`
+5 -5
View File
@@ -36,12 +36,11 @@ else
exit 1
fi
# 777: read/write for owner, group and every other account. Root (the display
# service) does not need it -- root ignores mode bits -- so the "other" bits
# only matter to accounts that are neither $REAL_USER nor root.
# Set permissions to allow read/write for owner, group, and others (for root service user)
# Note: 777 allows root (service user) to write, which is necessary when service runs as root
echo "Setting permissions for assets directory..."
if sudo chmod -R 777 "$ASSETS_DIR"; then
echo "✓ Set assets directory permissions to 777"
echo "✓ Set assets directory permissions to 777 (writable by root service user)"
else
echo "✗ Failed to set assets directory permissions"
exit 1
@@ -71,7 +70,8 @@ for SPORTS_DIR in "${SPORTS_DIRS[@]}"; do
echo " - Current permissions:"
ls -ld "$FULL_PATH"
# Owned by the real user; 777 as above (root can write here regardless)
# Ensure the directory is writable by both the real user and root (service user)
# Use 777 permissions to allow root (service) to write, or set group ownership
sudo chmod 777 "$FULL_PATH"
sudo chown "$REAL_USER:$REAL_GROUP" "$FULL_PATH"
+59 -35
View File
@@ -1,23 +1,11 @@
#!/bin/bash
# LEDMatrix Cache Permissions Fix Script
#
# /var/cache/ledmatrix is shared by the display service (root) and the web
# interface (your user) through the ledmatrix group: root:ledmatrix, 2775,
# files 660. scripts/install/setup_cache.sh is what sets that up (the
# installer's Step 2 runs it, and install_web_service.sh keeps the group), so
# this script runs it rather than applying a model of its own. It used to set
# the directory 777 and re-group it to your own group, replacing the ledmatrix
# group everything else relies on.
#
# It also repairs ~/.ledmatrix_cache, the cache manager's fallback when
# /var/cache/ledmatrix is unusable.
# This script fixes permissions on all known cache directories so they're writable by the daemon or current user
# Also sets up placeholder logo directories for sports managers
echo "Fixing LEDMatrix cache directory permissions..."
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SETUP_CACHE="$SCRIPT_DIR/../install/setup_cache.sh"
# Get the real user (not root when running with sudo)
REAL_USER=${SUDO_USER:-$USER}
# Resolve the home directory of the real user robustly
@@ -28,36 +16,72 @@ else
fi
REAL_GROUP=$(id -gn "$REAL_USER")
# Known cache directories for LEDMatrix. Use the actual user's home instead of a hard-coded path.
CACHE_DIRS=(
"/var/cache/ledmatrix"
"$REAL_HOME/.ledmatrix_cache"
)
for CACHE_DIR in "${CACHE_DIRS[@]}"; do
echo ""
echo "Checking cache directory: $CACHE_DIR"
if [ ! -d "$CACHE_DIR" ]; then
echo " - Directory does not exist. Creating it..."
sudo mkdir -p "$CACHE_DIR"
fi
echo " - Current permissions:"
ls -ld "$CACHE_DIR"
echo " - Fixing permissions..."
# Make directory writable by services regardless of user context
sudo chmod 777 "$CACHE_DIR"
sudo chown "$REAL_USER":"$REAL_GROUP" "$CACHE_DIR"
echo " - Updated permissions:"
ls -ld "$CACHE_DIR"
echo " - Testing write access as $REAL_USER..."
if sudo -u "$REAL_USER" test -w "$CACHE_DIR"; then
echo " ✓ $CACHE_DIR is now writable by $REAL_USER"
else
echo " ✗ $CACHE_DIR is still not writable by $REAL_USER"
fi
echo " - Permissions fix complete for $CACHE_DIR."
done
# Set up placeholder logos directory for sports managers
echo ""
echo "Checking cache directory: /var/cache/ledmatrix"
if [ -f "$SETUP_CACHE" ]; then
bash "$SETUP_CACHE"
echo "Setting up placeholder logos directory for sports managers..."
PLACEHOLDER_DIR="/var/cache/ledmatrix/placeholder_logos"
if [ ! -d "$PLACEHOLDER_DIR" ]; then
echo "Creating placeholder logos directory: $PLACEHOLDER_DIR"
sudo mkdir -p "$PLACEHOLDER_DIR"
sudo chown "$REAL_USER":"$REAL_GROUP" "$PLACEHOLDER_DIR"
sudo chmod 777 "$PLACEHOLDER_DIR"
else
echo " ✗ $SETUP_CACHE not found; /var/cache/ledmatrix left unchanged."
echo "Placeholder logos directory already exists: $PLACEHOLDER_DIR"
sudo chmod 777 "$PLACEHOLDER_DIR"
sudo chown "$REAL_USER":"$REAL_GROUP" "$PLACEHOLDER_DIR"
fi
CACHE_DIR="$REAL_HOME/.ledmatrix_cache"
echo ""
echo "Checking cache directory: $CACHE_DIR"
if [ ! -d "$CACHE_DIR" ]; then
echo " - Directory does not exist. Creating it..."
sudo mkdir -p "$CACHE_DIR"
fi
echo " - Current permissions:"
ls -ld "$CACHE_DIR"
echo " - Fixing permissions..."
sudo chmod 777 "$CACHE_DIR"
sudo chown "$REAL_USER":"$REAL_GROUP" "$CACHE_DIR"
echo " - Updated permissions:"
ls -ld "$CACHE_DIR"
ls -ld "$PLACEHOLDER_DIR"
echo " - Testing write access as $REAL_USER..."
if sudo -u "$REAL_USER" test -w "$CACHE_DIR"; then
echo " ✓ $CACHE_DIR is now writable by $REAL_USER"
if sudo -u "$REAL_USER" test -w "$PLACEHOLDER_DIR"; then
echo " ✓ Placeholder logos directory is writable by $REAL_USER"
else
echo " ✗ $CACHE_DIR is still not writable by $REAL_USER"
echo " ✗ Placeholder logos directory is not writable by $REAL_USER"
fi
# Test with daemon user (which the system might run as)
if sudo -u daemon test -w "$PLACEHOLDER_DIR" 2>/dev/null; then
echo " ✓ Placeholder logos directory is writable by daemon user"
else
echo " ✗ Placeholder logos directory is not writable by daemon user"
fi
echo " - Permissions fix complete for $CACHE_DIR."
echo ""
echo "All cache directory permission fixes attempted."
echo "If you still see errors, check which user is running the LEDMatrix service and ensure it matches the owner above."
echo ""
echo "The system will now create placeholder logos in:"
echo " $PLACEHOLDER_DIR"
echo "This should eliminate the permission denied warnings for sports logos."
+5 -6
View File
@@ -51,10 +51,9 @@ fi
echo "Setting ownership to root:$ACTUAL_USER..."
sudo chown -R root:"$ACTUAL_USER" "$PLUGINS_DIR"
# Set directory permissions (2775: rwxrwsr-x)
# Owner (root) and group (ACTUAL_USER): read/write/execute, others: read/execute.
# The setgid bit makes new entries inherit the ACTUAL_USER group.
echo "Setting directory permissions to 2775 (rwxrwsr-x, setgid)..."
# Set directory permissions (775: rwxrwxr-x)
# Root: read/write/execute, Group (ACTUAL_USER): read/write/execute, Others: read/execute
echo "Setting directory permissions to 2775 (rwxrwxr-x + sticky bit)..."
find "$PLUGINS_DIR" -type d -exec sudo chmod 2775 {} \;
# Set file permissions (664: rw-rw-r--)
@@ -72,7 +71,7 @@ fi
echo "Setting ownership of plugin-repos to root:$ACTUAL_USER..."
sudo chown -R root:"$ACTUAL_USER" "$PLUGIN_REPOS_DIR"
echo "Setting plugin-repos directory permissions to 2775 (rwxrwsr-x, setgid)..."
echo "Setting plugin-repos directory permissions to 2775 (rwxrwxr-x + sticky bit)..."
find "$PLUGIN_REPOS_DIR" -type d -exec sudo chmod 2775 {} \;
echo "Setting plugin-repos file permissions to 664..."
@@ -88,7 +87,7 @@ echo "plugin-repos/:"
ls -la "$PLUGIN_REPOS_DIR" 2>/dev/null || echo " (empty or not accessible)"
echo ""
echo "Permissions summary:"
echo "- Root service: Can read/write plugins (as root it needs no permission bits)"
echo "- Root service: Can read/write plugins (for PWM hardware access)"
echo "- Web service ($ACTUAL_USER): Can read/write plugins (for installation)"
echo "- Others: Can read plugins"
+5 -51
View File
@@ -25,9 +25,8 @@ echo ""
echo "This script will:"
echo "1. Add the web user to the 'systemd-journal' group for log access"
echo "2. Add the web user to the 'adm' group for additional system access"
echo "3. Make the project directory yours again, keeping the root-owned sudo"
echo " helpers and config_secrets.json as the installer leaves them"
echo " (sudoers rules are configure_web_sudo.sh's job, not this script's)"
echo "3. Configure sudoers for passwordless access to system commands"
echo "4. Set proper file permissions"
echo ""
# Ask for confirmation
@@ -63,51 +62,6 @@ else
echo "✗ Failed to set project ownership"
fi
# The chown above also takes back two kinds of file that first_time_install.sh
# deliberately keeps from the web user. Put them back the way the installer
# leaves them (its Steps 11 and 11.1), whether or not the chown succeeded.
#
# 1. The helpers /etc/sudoers.d/ledmatrix_web lets the web user run as root
# (scripts/install/lib_sudoers.sh). A copy the web user owns is a root shell
# for whoever can edit it, so they stay root-owned and writable by root only.
# Keep this list in step with the installer's Step 11.1 loop;
# test/test_web_sudoers_installers_agree.py checks both against the grants.
for helper in safe_plugin_rm.sh safe_pip_install.sh; do
HELPER_PATH="$PROJECT_DIR/scripts/fix_perms/$helper"
if [ -f "$HELPER_PATH" ]; then
if sudo chown root:root "$HELPER_PATH" && sudo chmod 755 "$HELPER_PATH"; then
echo "✓ $helper is root-owned again (sudo runs it as root)"
else
echo "⚠ Could not make $HELPER_PATH root-owned, mode 755."
echo " Fix it by hand: sudo chown root:root $HELPER_PATH && sudo chmod 755 $HELPER_PATH"
fi
fi
done
# 2. config_secrets.json: owned by the account ledmatrix-web.service runs as,
# group ledmatrix, mode 640 -- the same owner, group and mode as the
# installer's Step 11 gives it.
SECRETS_FILE="$PROJECT_DIR/config/config_secrets.json"
if [ -f "$SECRETS_FILE" ]; then
SECRETS_OWNER=""
if [ -f /etc/systemd/system/ledmatrix-web.service ]; then
SECRETS_OWNER=$(grep -m1 "^User=" /etc/systemd/system/ledmatrix-web.service | cut -d'=' -f2 || true)
fi
SECRETS_OWNER="${SECRETS_OWNER:-$WEB_USER}"
if getent group ledmatrix >/dev/null 2>&1; then
SECRETS_OWNERSHIP="$SECRETS_OWNER:ledmatrix"
else
# No ledmatrix group means the installer never ran; keep the chown's group.
SECRETS_OWNERSHIP="$SECRETS_OWNER"
fi
if sudo chown "$SECRETS_OWNERSHIP" "$SECRETS_FILE" && sudo chmod 640 "$SECRETS_FILE"; then
echo "✓ config_secrets.json restored to $SECRETS_OWNERSHIP, mode 640"
else
echo "⚠ Could not restore $SECRETS_FILE to $SECRETS_OWNERSHIP, mode 640."
echo " Fix it by hand: sudo chown $SECRETS_OWNERSHIP $SECRETS_FILE && sudo chmod 640 $SECRETS_FILE"
fi
fi
# Set proper permissions for config files
if sudo chmod 644 "$PROJECT_DIR/config/config.json" 2>/dev/null; then
echo "✓ Set config file permissions"
@@ -132,7 +86,7 @@ echo "Step 5: Testing sudo access..."
if sudo -n systemctl status ledmatrix.service > /dev/null 2>&1; then
echo "✓ Sudo access test passed"
else
echo "⚠ Sudo access test failed - you may need to run scripts/install/configure_web_sudo.sh"
echo "⚠ Sudo access test failed - you may need to run configure_web_sudo.sh"
fi
echo ""
@@ -147,5 +101,5 @@ echo ""
echo "After logging back in, test journal access with:"
echo " journalctl --no-pager --lines=5"
echo ""
echo "If you still have sudo issues, run (as this user, without sudo):"
echo " $PROJECT_DIR/scripts/install/configure_web_sudo.sh"
echo "If you still have sudo issues, run:"
echo " ./configure_web_sudo.sh"
-389
View File
@@ -1,389 +0,0 @@
#!/usr/bin/env python3
"""Soak a running display and report how often moving frames reached the panel late.
Runs NEXT TO the display service, as any user: it only reads the stats file the
service writes (src/common/frame_timing.py) at the start and end of the run and
reports the difference. Nothing is stopped, restarted or drawn.
# 10 minutes, as the display is now
python3 scripts/frame_soak.py
# the same with the web preview open (the preview's PNG encodes are one of
# the things that used to make the render loop miss refreshes)
python3 scripts/frame_soak.py --preview
# quick look at the totals since the service started
python3 scripts/frame_soak.py --show
# keep the report for a before/after comparison
python3 scripts/frame_soak.py --duration 600 --json soak-before.json
Exit status: 0 when the late-frame rate is within ``--max-late-pct``, 1 when it
is not, 2 when there was nothing to measure (no stats file, the service
restarted mid-run, or nothing scrolled).
What the numbers mean
---------------------
late frames frames that reached the panel one or more refreshes after they
were due -- the panel showed the previous frame again, which on
a moving strip is a visible hitch. This is the pass/fail number.
freezes gaps of 250ms+ inside a scroll: recomposes, plugin handovers,
blocking calls on the render thread. Reported, not failed on,
since some are handovers between plugins rather than faults.
blit copying the frame into the matrix canvas (rgbmatrix SetImage).
Grows with width x height x pwm_bits.
wait blocked in SwapOnVSync, i.e. slack before the refresh.
work everything else between two frames: drawing, scrolling, and
waiting for the GIL.
"""
from __future__ import annotations
import argparse
import json
import os
import sys
import time
from pathlib import Path
from typing import Any, Dict, Optional
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.common.frame_timing import ( # noqa: E402
BUCKET_COUNT,
SCHEMA_VERSION,
default_stats_path,
)
#: Touched by the web UI while someone has the preview open; a fresh marker
#: puts the display service's snapshot writer at full rate. Same path as
#: DisplayManager._viewer_marker_path.
VIEWER_MARKER = "/tmp/led_matrix_preview_viewer" # nosec B108 - fixed path shared with the service
#: A stats file not rewritten for this long means nothing is being presented.
STALE_SECONDS = 30.0
def load(path: str) -> Optional[Dict[str, Any]]:
try:
with open(path, encoding="utf-8") as handle:
stats = json.load(handle)
except (OSError, ValueError):
return None
if not isinstance(stats, dict) or stats.get("version") != SCHEMA_VERSION:
return None
return stats
def _histogram(stats: Dict[str, Any], name: str) -> Dict[int, int]:
raw = (stats.get("histograms") or {}).get(name) or {}
return {int(k): int(v) for k, v in raw.items()}
def diff(before: Dict[str, Any], after: Dict[str, Any]) -> Dict[str, Any]:
"""What happened between two snapshots of the same process."""
tb, ta = before["totals"], after["totals"]
totals = {}
for key, value in ta.items():
if isinstance(value, dict):
totals[key] = {k: v - tb.get(key, {}).get(k, 0)
for k, v in value.items()}
elif key == "worst_interval_ms":
# A running maximum can't be differenced; it is reported as the
# worst since the service started.
totals[key] = value
else:
totals[key] = value - tb.get(key, 0)
histograms = {}
for name in (after.get("histograms") or {}):
hb, ha = _histogram(before, name), _histogram(after, name)
histograms[name] = {k: v - hb.get(k, 0) for k, v in ha.items()
if v - hb.get(k, 0) > 0}
return {"totals": totals, "histograms": histograms,
"seconds": after["updated"] - before["updated"]}
def percentiles(histogram: Dict[int, int], bucket_ms: float) -> Dict[str, Any]:
"""p50/p95/p99/max from a sparse histogram, as each bucket's upper edge."""
count = sum(histogram.values())
if not count:
return {}
out = {}
targets = {"p50": 0.50, "p95": 0.95, "p99": 0.99}
running = 0
for index in sorted(histogram):
running += histogram[index]
for name, fraction in list(targets.items()):
if running >= fraction * count:
out[name] = _edge(index, bucket_ms)
del targets[name]
out["max"] = _edge(max(histogram), bucket_ms)
return out
def _edge(index: int, bucket_ms: float):
if index >= BUCKET_COUNT - 1:
return f">={index * bucket_ms:g}"
return round((index + 1) * bucket_ms, 2)
def build_report(before, after, preview: bool) -> Dict[str, Any]:
delta = diff(before, after)
totals = delta["totals"]
frames = totals["scroll_frames"]
# The rates are over frames judged against a known refresh period. Stats
# from a recorder that predates the count fall back to every frame.
timed = totals.get("timed_frames", frames) if "timed_frames" in totals else frames
hours = delta["seconds"] / 3600.0 if delta["seconds"] > 0 else 0.0
bucket_ms = after.get("bucket_ms", 0.25)
report = {
"seconds": round(delta["seconds"], 1),
"preview": preview,
"info": after.get("info"),
"binding_releases_gil": after.get("binding_releases_gil"),
"measured_refresh_hz": after.get("measured_refresh_hz"),
"scroll_frames": frames,
"static_frames": totals["static_frames"],
"late_frames": totals["late_frames"],
"timed_frames": timed,
"late_pct": round(100.0 * totals["late_frames"] / timed, 3) if timed else None,
"missed_refreshes": totals["missed_refreshes"],
"late_by": totals["late_by"],
"early_frames": totals.get("early_frames", 0),
"early_pct": (round(100.0 * totals.get("early_frames", 0) / timed, 3)
if timed else None),
"freeze_by": totals.get("freeze_by", {}),
"freezes": totals["freezes"],
"freezes_per_hour": round(totals["freezes"] / hours, 1) if hours else None,
"freeze_seconds": round(totals["freeze_seconds"], 2),
"worst_interval_ms": (round(totals["worst_interval_ms"], 1)
if totals["worst_interval_ms"] else None),
"timing_ms": {name: percentiles(h, bucket_ms)
for name, h in delta["histograms"].items()},
}
# The rate the panel held while rendering: the typical frame's interval
# per refresh held. A few percent under the idle rate is normal (the Pi is
# bit-banging the panel and pushing frames at once); a widening gap between
# the two is a render-cost regression even when nothing is late.
typical = (report["timing_ms"].get("interval_per_hold") or {}).get("p50")
# percentiles() reports a bucket's upper edge; the midpoint is the better
# estimate, and half a 0.25ms bucket is already ~1% at 100Hz -- the size
# of the idle-vs-held gap this number exists to show.
if isinstance(typical, (int, float)) and typical > bucket_ms / 2:
report["held_refresh_hz"] = round(1000.0 / (typical - bucket_ms / 2), 1)
else:
report["held_refresh_hz"] = None
return report
def print_report(report: Dict[str, Any], limit: float) -> None:
info = report.get("info") or {}
size = "{}x{}".format(
(info.get("cols") or 0) * (info.get("chain_length") or 1),
(info.get("rows") or 0) * (info.get("parallel") or 1))
gil = {True: "releases the GIL", False: "STOCK (holds the GIL in SwapOnVSync)",
None: "unknown"}[report.get("binding_releases_gil")]
print(f"Rig {info.get('pi_model') or 'unknown'}")
print(f"Panel {size} chain {info.get('chain_length')} x parallel "
f"{info.get('parallel')} pwm_bits {info.get('pwm_bits')} "
f"slowdown {info.get('gpio_slowdown')} mapping {info.get('hardware_mapping')}")
print(f"Refresh {report.get('measured_refresh_hz') or '?'} Hz measured, "
f"cap {info.get('limit_refresh_rate_hz')}")
print(f"Binding {gil}")
print(f"Run {report['seconds']:.0f}s, preview "
f"{'open (simulated)' if report['preview'] else 'as-is'}")
print()
frames = report["scroll_frames"]
print(f"Scrolling frames {frames}")
if frames:
late_by = report["late_by"]
print(f"Late frames {report['late_frames']} ({report['late_pct']}%)"
f" missed refreshes {report['missed_refreshes']}"
f" [by 1: {late_by['1']}, 2: {late_by['2']}, "
f"3-5: {late_by['3-5']}, 6+: {late_by['6+']}]")
if report["early_frames"]:
print(f"Early frames {report['early_frames']} "
f"({report['early_pct']}%) swaps returned a refresh early")
print(f"Freezes >=250ms {report['freezes']}"
f" ({report['freezes_per_hour']}/h, {report['freeze_seconds']}s total)"
f" worst gap since start {report['worst_interval_ms'] or '-'} ms")
if report["freezes"]:
print(" by length: " + ", ".join(
f"{k}: {v}" for k, v in report["freeze_by"].items()))
print()
print(f"{'ms':<18}{'p50':>8}{'p95':>8}{'p99':>8}{'max':>8}")
for name in ("blit", "wait", "work", "interval_per_hold"):
row = report["timing_ms"].get(name) or {}
print(f"{name:<18}" + "".join(f"{str(row.get(k, '-')):>8}"
for k in ("p50", "p95", "p99", "max")))
print()
if report["late_pct"] is None:
print("RESULT nothing scrolled - no verdict")
elif not locked(report, limit):
ceiling = refresh_ceiling(report)
if (report.get("early_pct") or 0.0) > limit:
why = (f"{report['early_pct']}% of frames came a refresh early, so the "
"swaps were not waiting for the panel")
else:
why = (f"frames arrived at {report['measured_refresh_hz']}Hz, faster than "
f"the panel can refresh ({ceiling:g}Hz)")
print(f"RESULT FAIL NOT LOCKED: {why}, and the late count means nothing")
elif report["late_pct"] <= limit:
print(f"RESULT PASS {report['late_pct']}% late <= {limit}%")
else:
print(f"RESULT FAIL {report['late_pct']}% late > {limit}%")
#: How far over the panel's rate frames may arrive before the loop cannot have
#: been waiting for it. The margin covers the refresh wandering a little.
CEILING_MARGIN = 1.05
def refresh_ceiling(report: Dict[str, Any]) -> Optional[float]:
"""The fastest the panel can refresh, as far as this run knows.
The benchmark measures it (``idle_refresh_hz``); the service only knows its
cap. With neither, there is no ceiling to check against.
"""
idle = report.get("idle_refresh_hz")
if idle:
return float(idle)
cap = (report.get("info") or {}).get("limit_refresh_rate_hz")
try:
cap = float(cap)
except (TypeError, ValueError):
return None
return cap if cap > 0 else None
def locked(report: Dict[str, Any], limit: float) -> bool:
"""Whether the loop was paced by the panel at all.
Two ways it is not. Frames a whole refresh early mean some swaps did not
wait. And a loop that never waited at all -- the dirty-tracking skip firing
mid-scroll let one free-run at 827fps -- looks self-consistent to a refresh
estimate taken from its own frames, so nothing registers as early; what
gives it away is a "refresh" faster than the panel can physically do.
"""
if (report.get("early_pct") or 0.0) > limit:
return False
ceiling = refresh_ceiling(report)
measured = report.get("measured_refresh_hz")
return not (ceiling and measured and measured > ceiling * CEILING_MARGIN)
def passed(report: Dict[str, Any], limit: float) -> bool:
return (report["late_pct"] is not None and locked(report, limit)
and report["late_pct"] <= limit)
def touch_marker() -> bool:
try:
with open(VIEWER_MARKER, "a"):
pass
os.utime(VIEWER_MARKER, None)
return True
except OSError:
return False
def wait_for_fresh(path: str, timeout: float) -> Optional[Dict[str, Any]]:
"""The first snapshot written after now, so both ends of the run are exact."""
first = load(path)
deadline = time.time() + timeout
while time.time() < deadline:
current = load(path)
if current and (first is None or current["updated"] != first["updated"]):
return current
time.sleep(0.5)
return None
def main(argv=None) -> int:
parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
parser.add_argument("--duration", type=float, default=600.0,
help="seconds to soak (default 600)")
parser.add_argument("--preview", action="store_true",
help="keep the web-preview viewer marker fresh, as an "
"open preview tab does")
parser.add_argument("--max-late-pct", type=float, default=0.1,
help="fail above this percentage of late frames (default 0.1)")
parser.add_argument("--stats", default=default_stats_path(),
help="stats file written by the display service")
parser.add_argument("--json", metavar="PATH",
help="also write the report as JSON")
parser.add_argument("--show", action="store_true",
help="print totals since the service started and exit")
args = parser.parse_args(argv)
current = load(args.stats)
if current is None:
print(f"No frame stats at {args.stats}. Is the display service running a "
"build with frame timing, and has anything scrolled for ~10s?",
file=sys.stderr)
return 2
if time.time() - current["updated"] > STALE_SECONDS:
print(f"Frame stats are {time.time() - current['updated']:.0f}s old: nothing "
"has been presented recently (static screen, or the service stopped).",
file=sys.stderr)
if not args.show:
return 2
if args.show:
empty = json.loads(json.dumps(current))
for key, value in empty["totals"].items():
empty["totals"][key] = ({k: 0 for k in value} if isinstance(value, dict)
else 0)
empty["histograms"] = {}
empty["updated"] = current["started"]
report = build_report(empty, current, preview=False)
print_report(report, args.max_late_pct)
return 0
if args.preview and not touch_marker():
print(f"Cannot touch {VIEWER_MARKER}; run as the web service's user to "
"simulate an open preview.", file=sys.stderr)
return 2
print(f"Waiting for a fresh baseline from {args.stats} ...", flush=True)
before = wait_for_fresh(args.stats, timeout=60.0)
if before is None:
print("The stats file stopped updating.", file=sys.stderr)
return 2
end = time.time() + args.duration
next_progress = time.time() + 60.0
while time.time() < end:
if args.preview:
touch_marker()
time.sleep(1.0)
if time.time() >= next_progress:
now = load(args.stats)
if now and now.get("pid") == before["pid"]:
done = now["totals"]["scroll_frames"] - before["totals"]["scroll_frames"]
late = now["totals"]["late_frames"] - before["totals"]["late_frames"]
print(f" {int(end - time.time())}s left: {done} scrolling frames, "
f"{late} late", flush=True)
next_progress += 60.0
after = wait_for_fresh(args.stats, timeout=60.0)
if after is None:
print("The stats file stopped updating during the run.", file=sys.stderr)
return 2
if after.get("pid") != before.get("pid"):
print("The display service restarted during the run; results discarded.",
file=sys.stderr)
return 2
report = build_report(before, after, preview=args.preview)
print()
print_report(report, args.max_late_pct)
if args.json:
with open(args.json, "w", encoding="utf-8") as handle:
json.dump(report, handle, indent=2)
if report["late_pct"] is None:
return 2
return 0 if passed(report, args.max_late_pct) else 1
if __name__ == "__main__":
sys.exit(main())
-15
View File
@@ -19,21 +19,6 @@ This directory contains scripts for installing and configuring the LEDMatrix sys
(the user who runs the script, i.e. the one you installed LEDMatrix as;
there is no `ledmatrix` system user) the passwordless `nmcli` and related
WiFi permissions the web interface needs
- **`install_dns_fix.sh`** - Optional. Installs `ledmatrix-dns-fix.service`,
which adds `options single-request` to the resolver when API calls time
out (see `systemd/README.md`)
- **`install_mqtt_bridge.sh`** - Optional. Installs the Home Assistant MQTT
bridge service (see `integrations/mqtt_bridge/README.md`)
Libraries (sourced, not run):
- **`lib_sudoers.sh`** - The web interface's sudo allow-list
(`/etc/sudoers.d/ledmatrix_web`), shared by `first_time_install.sh` and
`configure_web_sudo.sh`
- **`lib_systemd_render.sh`** - `sed_escape_replacement`, used by every
script that renders a unit from `systemd/*.service`
- **`lib_lowmem.sh`** - Build-job sizing and temporary swap for the C++
build on low-memory Pis (`first_time_install.sh` Step 6)
## Usage
+69 -64
View File
@@ -26,31 +26,17 @@ fi
# Get the full paths to commands and validate each one
MISSING_CMDS=()
# Full path of a command, also looking in the sbin directories. This script runs
# as the web user, whose PATH usually lacks /usr/sbin and /sbin -- where reboot
# and poweroff live -- so `command -v` alone silently dropped their rules.
find_command() {
local found
found=$(command -v "$1" 2>/dev/null) && { printf '%s\n' "$found"; return 0; }
for dir in /usr/sbin /sbin /usr/bin /bin; do
if [ -x "$dir/$1" ]; then
printf '%s\n' "$dir/$1"
return 0
fi
done
return 1
}
SYSTEMCTL_PATH=$(find_command systemctl) || true
REBOOT_PATH=$(find_command reboot) || true
POWEROFF_PATH=$(find_command poweroff) || true
BASH_PATH=$(find_command bash) || true
JOURNALCTL_PATH=$(find_command journalctl) || true
PYTHON_PATH=$(command -v python3) || true
SYSTEMCTL_PATH=$(command -v systemctl) || true
REBOOT_PATH=$(command -v reboot) || true
POWEROFF_PATH=$(command -v poweroff) || true
BASH_PATH=$(command -v bash) || true
JOURNALCTL_PATH=$(command -v journalctl) || true
SAFE_RM_PATH="$PROJECT_ROOT/scripts/fix_perms/safe_plugin_rm.sh"
SAFE_PIP_INSTALL_PATH="$PROJECT_ROOT/scripts/fix_perms/safe_pip_install.sh"
# Validate required commands (systemctl and bash are essential)
for CMD_NAME in SYSTEMCTL_PATH BASH_PATH; do
# Validate required commands (systemctl, bash, python3 are essential)
for CMD_NAME in SYSTEMCTL_PATH BASH_PATH PYTHON_PATH; do
CMD_VAL="${!CMD_NAME}"
if [ -z "$CMD_VAL" ]; then
MISSING_CMDS+=("$CMD_NAME")
@@ -73,17 +59,8 @@ if [ ! -f "$SAFE_PIP_INSTALL_PATH" ]; then
exit 1
fi
# The rules are shared with first_time_install.sh (Step 10) so the two cannot
# drift apart; add or remove a grant in lib_sudoers.sh, not here.
SUDOERS_LIB="$PROJECT_DIR/lib_sudoers.sh"
if [ ! -f "$SUDOERS_LIB" ]; then
echo "Error: Sudoers rules library not found: $SUDOERS_LIB" >&2
exit 1
fi
# shellcheck source=scripts/install/lib_sudoers.sh
. "$SUDOERS_LIB"
echo "Command paths:"
echo " Python: $PYTHON_PATH"
echo " Systemctl: $SYSTEMCTL_PATH"
echo " Reboot: ${REBOOT_PATH:-(not found, skipping)}"
echo " Poweroff: ${POWEROFF_PATH:-(not found, skipping)}"
@@ -92,24 +69,62 @@ echo " Journalctl: ${JOURNALCTL_PATH:-(not found, skipping)}"
echo " Safe plugin rm: $SAFE_RM_PATH"
echo " Safe pip install: $SAFE_PIP_INSTALL_PATH"
# Create a temporary sudoers file. A predictable name in a world-writable
# directory is a symlink target, and these rules end up in /etc/sudoers.d, so
# let mktemp pick the name; the trap removes it however the script ends.
TEMP_SUDOERS=$(mktemp "${TMPDIR:-/tmp}/ledmatrix_web_sudoers.XXXXXX") || {
echo "Error: could not create a temporary file" >&2
exit 1
}
trap 'rm -f "$TEMP_SUDOERS"' EXIT
# Create a temporary sudoers file
TEMP_SUDOERS="/tmp/ledmatrix_web_sudoers_$$"
web_sudoers_rules "$WEB_USER" "$PROJECT_ROOT" "$SYSTEMCTL_PATH" "$BASH_PATH" \
"$REBOOT_PATH" "$POWEROFF_PATH" "$JOURNALCTL_PATH" > "$TEMP_SUDOERS"
{
echo "# LED Matrix Web Interface passwordless sudo configuration"
echo "# This allows the web interface user to run specific commands without a password"
echo ""
echo "# Allow $WEB_USER to run specific commands without a password for the LED Matrix web interface"
# Optional: reboot/poweroff (non-critical — skip if not found)
if [ -n "$REBOOT_PATH" ]; then
echo "$WEB_USER ALL=(ALL) NOPASSWD: $REBOOT_PATH"
fi
if [ -n "$POWEROFF_PATH" ]; then
echo "$WEB_USER ALL=(ALL) NOPASSWD: $POWEROFF_PATH"
fi
# Required: systemctl
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH enable ledmatrix.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH disable ledmatrix.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH status ledmatrix.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix-web.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix-web.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web.service"
# Optional: journalctl (non-critical — skip if not found)
#
# NOEXEC, matching first_time_install.sh. These rules end in a wildcard and
# journalctl starts a pager, so without it the caller can reach a shell:
# less runs "!command" as the user the pager belongs to, which here is
# root. NOEXEC stops the granted command executing anything of its own.
if [ -n "$JOURNALCTL_PATH" ]; then
echo "$WEB_USER ALL=(ALL) NOPASSWD:NOEXEC: $JOURNALCTL_PATH -u ledmatrix.service *"
echo "$WEB_USER ALL=(ALL) NOPASSWD:NOEXEC: $JOURNALCTL_PATH -u ledmatrix *"
echo "$WEB_USER ALL=(ALL) NOPASSWD:NOEXEC: $JOURNALCTL_PATH -t ledmatrix *"
fi
echo ""
echo "# Allow web user to remove plugin directories via vetted helper script"
echo "# The helper validates that the target path resolves inside plugin-repos/ or plugins/"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $SAFE_RM_PATH *"
echo ""
echo "# Allow web user to install a plugin's requirements.txt as root via vetted"
echo "# helper script, so packages are visible to root-run ledmatrix.service"
echo "# (not just the web interface's own user). The helper validates the target"
echo "# is requirements.txt at the project root or under plugin-repos/ or plugins/."
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $SAFE_PIP_INSTALL_PATH *"
} > "$TEMP_SUDOERS"
# Never offer to install rules we have not parsed. A malformed drop-in in
# /etc/sudoers.d makes sudo refuse every command for every user.
# visudo lives in /usr/sbin, which is not on every user's PATH.
if ! command -v visudo >/dev/null 2>&1 && [ -x /usr/sbin/visudo ]; then
PATH="$PATH:/usr/sbin"
fi
if command -v visudo >/dev/null 2>&1; then
if ! visudo -c -f "$TEMP_SUDOERS" >/dev/null 2>&1; then
echo ""
@@ -119,8 +134,6 @@ if command -v visudo >/dev/null 2>&1; then
rm -f "$TEMP_SUDOERS"
exit 1
fi
else
echo "⚠ visudo not found; the rules below have not been validated"
fi
echo ""
@@ -149,7 +162,7 @@ if [[ ! $REPLY =~ ^[Yy]$ ]]; then
exit 0
fi
# Apply the configuration
# Apply the configuration using visudo
echo "Applying sudoers configuration..."
# Harden the helper script: root-owned, not writable by web user
echo "Hardening safe_plugin_rm.sh ownership..."
@@ -168,29 +181,21 @@ if ! sudo chmod 755 "$SAFE_PIP_INSTALL_PATH"; then
fi
if sudo cp "$TEMP_SUDOERS" /etc/sudoers.d/ledmatrix_web; then
# sudo reads /etc/sudoers.d files that are root-owned and not writable by
# group or other; 440 is the mode visudo and first_time_install.sh use.
if ! sudo chmod 440 /etc/sudoers.d/ledmatrix_web; then
echo "Warning: could not set mode 440 on /etc/sudoers.d/ledmatrix_web"
fi
echo "Configuration applied successfully!"
echo ""
echo "Testing sudo access..."
# Ask sudo whether two of the new rules let this user in without a
# password. `sudo -l CMD` answers from the rules without running CMD, so
# this does not depend on whether ledmatrix.service is running, and it
# tests commands the rules actually grant.
if sudo -n -l "$SYSTEMCTL_PATH" status ledmatrix.service > /dev/null 2>&1; then
# Test a few commands
if sudo -n systemctl status ledmatrix.service > /dev/null 2>&1; then
echo "✓ systemctl status ledmatrix.service - OK"
else
echo "✗ systemctl status ledmatrix.service - not allowed without a password"
echo "✗ systemctl status ledmatrix.service - Failed"
fi
if sudo -n -l "$BASH_PATH" "$SAFE_RM_PATH" "$PROJECT_ROOT/plugin-repos/example" > /dev/null 2>&1; then
echo "✓ safe_plugin_rm.sh helper - OK"
if sudo -n test -f "$PROJECT_ROOT/start_display.sh"; then
echo "✓ File access test - OK"
else
echo "✗ safe_plugin_rm.sh helper - not allowed without a password"
echo "✗ File access test - Failed"
fi
echo ""
+4 -27
View File
@@ -144,11 +144,6 @@ $WEB_USER ALL=(ALL) NOPASSWD: $MKDIR_PATH -p /etc/NetworkManager/dnsmasq-shared.
$WEB_USER ALL=(ALL) NOPASSWD: /usr/bin/cp /tmp/hostapd.conf /etc/hostapd/hostapd.conf
$WEB_USER ALL=(ALL) NOPASSWD: /usr/bin/cp /tmp/dnsmasq.conf /etc/dnsmasq.d/ledmatrix-captive.conf
$WEB_USER ALL=(ALL) NOPASSWD: /usr/bin/rm -f /etc/dnsmasq.d/ledmatrix-captive.conf
# The same captive-portal DNS drop-in for NetworkManager's shared-mode dnsmasq
# (wifi_manager._write_nm_dnsmasq_captive_conf / _remove_nm_dnsmasq_captive_conf),
# exact paths.
$WEB_USER ALL=(ALL) NOPASSWD: /usr/bin/cp /tmp/ledmatrix-nm-dnsmasq.conf /etc/NetworkManager/dnsmasq-shared.d/ledmatrix-captive.conf
$WEB_USER ALL=(ALL) NOPASSWD: /usr/bin/rm -f /etc/NetworkManager/dnsmasq-shared.d/ledmatrix-captive.conf
EOF
echo "Generated sudoers configuration:"
@@ -156,21 +151,6 @@ echo "--------------------------------"
cat "$TEMP_SUDOERS"
echo "--------------------------------"
# Never install rules we have not parsed. A malformed drop-in in
# /etc/sudoers.d makes sudo refuse every command for every user, which on a
# headless Pi leaves no way in at all. first_time_install.sh and
# configure_web_sudo.sh check their rules the same way.
if command -v visudo >/dev/null 2>&1; then
if ! visudo -c -f "$TEMP_SUDOERS" >/dev/null 2>&1; then
echo "✗ The generated sudoers rules did not parse:" >&2
visudo -c -f "$TEMP_SUDOERS" >&2 || true
echo " Leaving $SUDOERS_FILE unchanged." >&2
exit 1
fi
else
echo "⚠ visudo not found; installing the sudoers rules unvalidated"
fi
# Apply the sudoers configuration
echo ""
echo "Applying sudoers configuration..."
@@ -233,14 +213,11 @@ rm -f "$TEMP_POLKIT"
echo ""
echo "Step 3: Testing permissions..."
# Ask sudo whether one of the new rules lets this user in without a password.
# `sudo -l CMD` answers from the rules without running CMD, so the radio is
# left alone. (This used to run `nmcli device status`, which is not granted,
# so it could only ever report a failure.)
if sudo -n -l "$NMCLI_PATH" radio wifi on > /dev/null 2>&1; then
echo "✓ nmcli radio wifi on - OK"
# Test sudo access
if sudo -n "$NMCLI_PATH" device status > /dev/null 2>&1; then
echo "✓ nmcli device status - OK"
else
echo "✗ nmcli radio wifi on - not allowed without a password"
echo "✗ nmcli device status - Failed (this is expected if not connected)"
fi
echo ""
+1 -6
View File
@@ -10,10 +10,6 @@
set -e
PROJECT_ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
# shellcheck source=scripts/install/lib_systemd_render.sh
source "$PROJECT_ROOT_DIR/scripts/install/lib_systemd_render.sh"
SERVICE_NAME="ledmatrix-dns-fix"
UNIT_SRC="$PROJECT_ROOT_DIR/systemd/$SERVICE_NAME.service"
UNIT_DEST="/etc/systemd/system/$SERVICE_NAME.service"
@@ -38,8 +34,7 @@ fi
chmod +x "$PROJECT_ROOT_DIR/scripts/utils/apply_dns_single_request.sh"
echo "Installing $UNIT_DEST..."
ESCAPED_PROJECT_ROOT_DIR=$(sed_escape_replacement "$PROJECT_ROOT_DIR")
sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g" "$UNIT_SRC" \
sed "s|__PROJECT_ROOT_DIR__|$PROJECT_ROOT_DIR|g" "$UNIT_SRC" \
| $SUDO tee "$UNIT_DEST" > /dev/null
# Order ledmatrix.service after the fix. `Before=` in the unit itself only
+1 -6
View File
@@ -9,10 +9,6 @@
set -e
PROJECT_ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
# shellcheck source=scripts/install/lib_systemd_render.sh
source "$PROJECT_ROOT_DIR/scripts/install/lib_systemd_render.sh"
BRIDGE_DIR="$PROJECT_ROOT_DIR/integrations/mqtt_bridge"
SERVICE_NAME="ledmatrix-mqtt-bridge"
UNIT_SRC="$PROJECT_ROOT_DIR/systemd/$SERVICE_NAME.service"
@@ -44,8 +40,7 @@ python3 -m pip install -r "$BRIDGE_DIR/requirements.txt" 2>/dev/null \
|| python3 -m pip install --break-system-packages -r "$BRIDGE_DIR/requirements.txt"
echo "Installing $UNIT_DEST..."
ESCAPED_PROJECT_ROOT_DIR=$(sed_escape_replacement "$PROJECT_ROOT_DIR")
sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g" "$UNIT_SRC" \
sed "s|__PROJECT_ROOT_DIR__|$PROJECT_ROOT_DIR|g" "$UNIT_SRC" \
| $SUDO tee "$UNIT_DEST" > /dev/null
$SYSTEMCTL_CMD daemon-reload
+1 -6
View File
@@ -51,25 +51,20 @@ if [ ${#MISSING_PACKAGES[@]} -gt 0 ]; then
# Install packages automatically (no prompt)
# Use apt directly if running as root, otherwise use sudo
PACKAGES_OK=true
if [ "$EUID" -eq 0 ]; then
apt update || echo "⚠ apt update failed, continuing anyway..."
apt install -y "${MISSING_PACKAGES[@]}" || {
PACKAGES_OK=false
echo "⚠ Package installation failed, but continuing with WiFi monitor setup"
echo " You may need to install packages manually: apt install -y ${MISSING_PACKAGES[*]}"
}
else
sudo apt update || echo "⚠ apt update failed, continuing anyway..."
sudo apt install -y "${MISSING_PACKAGES[@]}" || {
PACKAGES_OK=false
echo "⚠ Package installation failed, but continuing with WiFi monitor setup"
echo " You may need to install packages manually: sudo apt install -y ${MISSING_PACKAGES[*]}"
}
fi
if [ "$PACKAGES_OK" = true ]; then
echo "✓ Package installation completed"
fi
echo "✓ Package installation completed"
fi
# Render the unit from systemd/ledmatrix-wifi-monitor.service rather than
-76
View File
@@ -1,76 +0,0 @@
#!/bin/bash
#
# The web interface's passwordless-sudo allow-list, /etc/sudoers.d/ledmatrix_web.
#
# Sourced by first_time_install.sh (Step 10) and
# scripts/install/configure_web_sudo.sh. Both used to carry their own copy of
# these rules, and the copies drifted: one granted safe_pip_install.sh and the
# other did not. Each caller still owns its own validate (visudo -c) / install /
# confirm flow; this file only prints the rules.
#
# Add or remove a grant here and nowhere else.
# web_sudoers_rules WEB_USER PROJECT_ROOT SYSTEMCTL_PATH BASH_PATH REBOOT_PATH POWEROFF_PATH JOURNALCTL_PATH
#
# Print the ledmatrix_web sudoers rules to stdout.
#
# SYSTEMCTL_PATH and BASH_PATH are required, and the caller must make sure they
# are not empty: `visudo -c` does not catch every such rule (with an empty
# BASH_PATH the helper rules still parse, granting the script itself).
# first_time_install.sh stops on a failed `which`; configure_web_sudo.sh checks
# them before calling this.
# REBOOT_PATH, POWEROFF_PATH and JOURNALCTL_PATH are optional: pass "" and
# their rules are left out.
web_sudoers_rules() {
local WEB_USER="${1:-}"
local PROJECT_ROOT="${2:-}"
local SYSTEMCTL_PATH="${3:-}"
local BASH_PATH="${4:-}"
local REBOOT_PATH="${5:-}"
local POWEROFF_PATH="${6:-}"
local JOURNALCTL_PATH="${7:-}"
cat << EOF
# LED Matrix Web Interface passwordless sudo configuration
# This allows the web interface user to run specific commands without a password
# Allow $WEB_USER to run specific commands without a password for the LED Matrix web interface
EOF
if [ -n "$REBOOT_PATH" ]; then
printf '%s\n' "$WEB_USER ALL=(ALL) NOPASSWD: $REBOOT_PATH"
fi
if [ -n "$POWEROFF_PATH" ]; then
printf '%s\n' "$WEB_USER ALL=(ALL) NOPASSWD: $POWEROFF_PATH"
fi
cat << EOF
$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix.service
$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix.service
$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix.service
$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH enable ledmatrix.service
$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH disable ledmatrix.service
$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH status ledmatrix.service
$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix
$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix.service
$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix-web.service
$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix-web.service
$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web.service
$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT/scripts/fix_perms/safe_plugin_rm.sh *
# Install a requirements.txt as root via vetted helper, so packages are visible
# to root-run ledmatrix.service (not just the web interface's own user).
$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT/scripts/fix_perms/safe_pip_install.sh *
EOF
if [ -n "$JOURNALCTL_PATH" ]; then
cat << EOF
# NOEXEC, because these rules end in a wildcard and journalctl starts a pager
# when its output is a terminal. From that pager (less) a "!sh" is a root
# shell -- the standard journalctl escalation. The web interface always passes
# --no-pager, so nothing here needs it, but the rule cannot require a flag that
# sits in the middle of the command line. NOEXEC stops the command executing
# another program at all, which closes the hole without depending on wildcard
# matching subtleties.
$WEB_USER ALL=(ALL) NOPASSWD:NOEXEC: $JOURNALCTL_PATH -u ledmatrix.service *
$WEB_USER ALL=(ALL) NOPASSWD:NOEXEC: $JOURNALCTL_PATH -u ledmatrix *
$WEB_USER ALL=(ALL) NOPASSWD:NOEXEC: $JOURNALCTL_PATH -t ledmatrix *
EOF
fi
}
+3 -4
View File
@@ -2,10 +2,9 @@
#
# Shared helper for rendering systemd unit templates via sed.
#
# Sourced by install_service.sh, install_web_service.sh,
# install_wifi_monitor.sh, install_dns_fix.sh and install_mqtt_bridge.sh so
# every unit renderer escapes sed replacement text the same way instead of
# carrying its own copy of the fix.
# Sourced by install_service.sh, install_web_service.sh and
# install_wifi_monitor.sh so all three escape sed replacement text the same
# way instead of carrying three copies of the same fix.
# sed_escape_replacement VALUE
#
+22 -8
View File
@@ -205,6 +205,27 @@ check_sudo() {
print_success "Sudo access confirmed"
}
# Fix /tmp permissions if needed (common issue when running via curl | bash)
# Note: /tmp permission fixing is now done inline before running first_time_install.sh
# This function is kept for backward compatibility but not actively used
fix_tmp_permissions() {
CURRENT_STEP="TMP directory check"
# Only fix if /tmp is actually not writable (don't preemptively fix)
if [ ! -w /tmp ]; then
print_warning "/tmp is not writable, attempting to fix..."
if [ "$EUID" -eq 0 ]; then
chmod 1777 /tmp 2>/dev/null || true
else
sudo chmod 1777 /tmp 2>/dev/null || true
fi
fi
# Ensure TMPDIR is set correctly
if [ -z "${TMPDIR:-}" ] || [ ! -w "${TMPDIR:-/tmp}" ]; then
export TMPDIR=/tmp
fi
}
# Main installation function
main() {
print_step "LED Matrix One-Shot Installation"
@@ -408,13 +429,6 @@ main() {
print_step "Installation Complete!"
print_success "LED Matrix has been successfully installed!"
echo ""
# first_time_install.sh -y reboots as its last action, so by now the
# reboot is under way (unless LEDMATRIX_SKIP_REBOOT_PROMPT=1 was set).
if [ "${LEDMATRIX_SKIP_REBOOT_PROMPT:-0}" != "1" ]; then
echo "The installer has just started a reboot to finish setup, so this"
echo "session may disconnect now. Give the Pi a few minutes to come back, then:"
echo ""
fi
echo "Next steps:"
echo " 1. Configure your settings: sudo nano $REPO_DIR/config/config.json"
if command -v hostname >/dev/null 2>&1; then
@@ -435,7 +449,7 @@ main() {
else
echo " 2. Or use the web interface: http://<your-pi-ip>:5000"
fi
echo " 3. The display service starts on boot; to start it by hand: sudo systemctl start ledmatrix.service"
echo " 3. Start the service: sudo systemctl start ledmatrix.service"
echo ""
else
print_error "Main installation script exited with code $INSTALL_EXIT_CODE"
-404
View File
@@ -1,404 +0,0 @@
#!/usr/bin/env python3
"""Benchmark the render loop against the panel's real refresh rate.
The question this answers is the one that decides whether a rig ships: *does
every frame present on the refresh it was meant to?* It drives the production
path -- a real ``DisplayManager`` and ``ScrollHelper``, the same crisp speed
resolver every ticker uses -- scrolls a synthetic strip for a while, and grades
it with the same frame-timing recorder the display service uses
(``src.common.frame_timing``), printing the same report as
``scripts/frame_soak.py``. A run passes when the loop was genuinely locked to
the panel and no more than ``--max-late-pct`` percent of frames were late.
Where frame_soak.py measures the service as it runs -- live content, plugin
updates, the web preview -- this measures the hardware and the render path
with nothing else in the way, on content that is identical every run. That is
what makes it the tool for comparing rigs (a Pi 3 against a Pi 4, one HAT
against another) and for A/B testing a change to the render path.
# stop the service first; it owns the GPIO
sudo systemctl stop ledmatrix
sudo python3 scripts/render_bench.py # 60s, default speed
sudo python3 scripts/render_bench.py --seconds 600 # the 10-minute gate
sudo python3 scripts/render_bench.py --speed 50 # a slower, held speed
sudo python3 scripts/render_bench.py --busy 2 # with background load
sudo python3 scripts/render_bench.py --json /tmp/pi4.json
sudo systemctl start ledmatrix
Like scripts/scroll_speeds.py, this never starts or stops the service itself,
so a crash here can never leave the panel dark.
Exit status is 0 when the run clears the gate, 1 when it does not, and 2 when
the run could not be set up (no hardware, no root, unusable config) -- so a rig
that cannot be measured is never mistaken for a rig that passed.
"""
from __future__ import annotations
import argparse
import json
import logging
import os
import sys
import threading
import time
import zlib
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.common import frame_timing, scroll_config # noqa: E402
sys.path.insert(0, str(Path(__file__).resolve().parent))
import frame_soak # noqa: E402 (same report, same verdict as the soak)
REPO = Path(__file__).resolve().parent.parent
CONFIG = REPO / "config" / "config.json"
#: Long enough to average out a scheduler hiccup, short enough that nobody
#: skips running it. The shipping gate is --seconds 600.
DEFAULT_SECONDS = 60.0
#: Seconds spent timing bare swaps before the scroll starts. The measurement
#: has to settle, but every second here is a second not scrolling.
MEASURE_SECONDS = 4.0
#: Scrolling discarded before the graded run starts: the first frames carry
#: first-touch costs and the scrolling state settling.
WARMUP_SECONDS = 2.0
def load_config() -> dict:
"""The config the display service would run with."""
try:
from src.config_manager import ConfigManager
config = ConfigManager().config
if isinstance(config, dict) and config:
return config
except Exception as exc: # noqa: BLE001 - any failure means use the plain read
print(f"ConfigManager unavailable ({exc}); reading {CONFIG} directly",
file=sys.stderr)
# ConfigManager pulls in a lot; a plain read is enough to drive the panel
# and keeps the benchmark usable on a half-installed machine.
try:
with open(CONFIG, encoding="utf-8") as handle:
config = json.load(handle)
except (OSError, ValueError) as exc:
sys.exit(f"could not read {CONFIG}: {exc}")
if not isinstance(config, dict):
sys.exit(f"{CONFIG} is not a config object")
return config
def build_strip(width: int, height: int, label: str):
"""A marquee strip a few screens wide, with text and colour.
Deliberately not plain white text on black: how long ``SetImage`` takes
depends on how many subpixels are lit, so a strip that is mostly dark
flatters the panel and hides exactly the regression this benchmark exists
to catch.
"""
from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import load_truetype
font = None
for path, size in (
(str(REPO / "assets/fonts/PressStart2P-Regular.ttf"), max(8, height // 4)),
("/usr/share/fonts/truetype/dejavu/DejaVuSansMono-Bold.ttf", max(10, height // 2)),
):
try:
font = load_truetype(path, size)
break
except OSError:
continue
if font is None:
font = ImageFont.load_default()
text = f" {label} *** THE QUICK BROWN FOX JUMPS OVER THE LAZY DOG *** "
probe = ImageDraw.Draw(Image.new("RGB", (8, 8)))
box = probe.textbbox((0, 0), text, font=font)
text_width = max(1, box[2] - box[0])
text_height = box[3] - box[1]
reps = max(2, (width * 4) // text_width + 1)
strip = Image.new("RGB", (text_width * reps, height), (0, 0, 0))
draw = ImageDraw.Draw(strip)
draw.fontmode = "1" # the panel has no partial brightness; see DisplayManager
palette = [(255, 210, 60), (80, 200, 255), (255, 90, 90), (140, 255, 140)]
for i in range(reps):
left = i * text_width
# A filled block per repeat, so a meaningful share of the strip is lit.
draw.rectangle(
[left + 4, height - 4, left + text_width - 4, height - 2],
fill=palette[i % len(palette)],
)
draw.text((left, (height - text_height) // 2 - box[1]), text,
font=font, fill=palette[(i + 1) % len(palette)])
return strip
class BackgroundLoad:
"""Threads that imitate plugins updating while the panel scrolls.
Not a simulation of any particular plugin -- it is the shape of the work
that competes with the render loop for the GIL: decoding JSON, resizing an
image, compressing bytes. A render loop that only holds its pacing on an
idle machine is not shippable, and this is how that shows up.
"""
def __init__(self, workers: int) -> None:
self.workers = max(0, workers)
self._stop = threading.Event()
self._threads: list = []
def __enter__(self) -> "BackgroundLoad":
for index in range(self.workers):
thread = threading.Thread(
target=self._run, args=(index,), name=f"bench-load-{index}", daemon=True)
thread.start()
self._threads.append(thread)
return self
def __exit__(self, *exc_info) -> None:
self._stop.set()
for thread in self._threads:
thread.join(timeout=2.0)
def _run(self, index: int) -> None:
from PIL import Image
payload = json.dumps({"games": [{"id": n, "score": [n, n + 1],
"name": f"team {n}"} for n in range(200)]})
image = Image.new("RGB", (256, 64), (12, 34, 56))
while not self._stop.wait(0.25 + 0.05 * index):
json.loads(payload)
image.resize((128, 32), Image.LANCZOS)
zlib.compress(image.tobytes(), 1)
def main(argv=None) -> int:
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--seconds", type=float, default=DEFAULT_SECONDS,
help=f"how long to scroll for (default {DEFAULT_SECONDS:.0f}; "
"the shipping gate is 600)")
parser.add_argument("--speed", type=float, default=None,
help="requested px/s; snapped to the nearest speed the "
"panel can show in whole pixels (default: one pixel "
"per refresh)")
parser.add_argument("--hz", type=float, default=None,
help="skip the idle measurement and take this as the "
"panel's rate (for reproducing a rig's numbers)")
parser.add_argument("--busy", type=int, default=0, metavar="N",
help="run N background workers imitating plugin updates")
parser.add_argument("--max-late-pct", "--max-missed", dest="max_late_pct",
type=float, default=0.1, metavar="PCT",
help="fail above this percentage of late frames (default 0.1)")
parser.add_argument("--json", dest="json_path", default=None, metavar="PATH",
help="also write the report as JSON, for comparing rigs")
parser.add_argument("--label", default=None,
help="name for this run in the JSON report (default: hostname)")
args = parser.parse_args(argv)
# Everything the display service logs would otherwise land in the middle of
# the report; the benchmark's own output is the point. The stall watchdog
# is the exception: a stack dump naming what held a frame up belongs here.
logging.basicConfig(level=logging.ERROR, stream=sys.stderr)
logging.getLogger("src.common.frame_timing").setLevel(logging.WARNING)
if hasattr(os, "geteuid") and os.geteuid() != 0:
print("this needs root for GPIO access - rerun with sudo", file=sys.stderr)
return 2
config = load_config()
from src.common.scroll_helper import ScrollHelper
from src.display_manager import DisplayManager
try:
display = DisplayManager(config, suppress_test_pattern=True)
except Exception as exc:
print(f"could not open the display ({exc}).\n"
"If the display service is running it owns the GPIO - stop it "
"first:\n sudo systemctl stop ledmatrix", file=sys.stderr)
return 2
if getattr(display, "matrix", None) is None:
print("the display came up in fallback mode - there is no panel here to "
"measure, and a software loop's frame times say nothing about "
"vsync. Run this on a rig.", file=sys.stderr)
return 2
width, height = display.width, display.height
if args.hz is not None:
idle_hz = float(args.hz)
print(f"taking the panel's rate as {idle_hz:.1f}Hz (given, not measured)")
else:
print(f"measuring the panel for {MEASURE_SECONDS:.0f}s...", flush=True)
idle_hz = frame_timing.measure_refresh_hz(display.matrix, MEASURE_SECONDS)
if idle_hz <= 0:
print("the panel did not answer a swap; cannot measure it",
file=sys.stderr)
return 2
cap = scroll_config.refresh_hz_from_config(config)
note = (f" (cap is {cap:.0f}Hz)" if idle_hz < cap * 0.98
else " (at its configured cap)")
print(f"panel refreshes at {idle_hz:.1f}Hz{note}")
requested = args.speed if args.speed else idle_hz
# Configured through the shared resolver rather than by setting the helper
# up by hand, so the benchmark measures the engine every ticker runs on. A
# speed the bench reached some other way would be measuring something no
# plugin does.
helper = ScrollHelper(width, height)
settings = scroll_config.configure(
helper,
plugin_config={"scroll_pixels_per_second": requested},
global_config=config,
refresh_hz=idle_hz,
display_manager=display,
)
choice = settings.crisp
if choice is None:
print("the resolver did not snap to a whole-pixel speed; nothing to "
"grade against", file=sys.stderr)
return 2
print(f"asked for {requested:.1f} px/s -> {choice.describe()}")
helper.set_sub_pixel_scrolling(False)
helper.set_scrolling_image(
build_strip(width, height, f"{choice.pixels_per_second:.0f} px/s"))
# The display service's own recorder, owned outright here: never flushed to
# the service's stats file, drained exactly at the start and end of the
# graded run, and seeded with the idle rate so a loop that never locked
# (free-running, or stuck at a fraction of the refresh) shows as early or
# late frames instead of looking self-consistent.
recorder = frame_timing.FrameTimingRecorder(
flush_interval=float("inf"),
info=display._frame_timing_info(), # pylint: disable=protected-access
refresh_hz=idle_hz,
)
recorder.scrolling_now = display._scrolling_now # pylint: disable=protected-access
display.frame_timing = recorder
print(f"scrolling {width}x{height} for {args.seconds:.0f}s"
+ (f" with {args.busy} background worker(s)" if args.busy else "")
+ " ...", flush=True)
frames = 0
duplicates = 0
blanks = 0
restarts = 0
last_column = None
before = None
started = time.perf_counter()
run_started = None
try:
with BackgroundLoad(args.busy):
while True:
now = time.perf_counter()
if run_started is None and now - started >= WARMUP_SECONDS:
recorder.drain()
before = recorder.snapshot()
run_started = now
frames = duplicates = blanks = restarts = 0
if run_started is not None and now - run_started >= args.seconds:
break
helper.update_scroll_position()
if helper.is_scroll_complete():
# The helper parks at the end of the strip and stops
# advancing, exactly as it does under a plugin -- which
# then hands over to the next one. Here there is nothing
# to hand over to, so start the strip again. Without this
# the benchmark measures a still image for the rest of the
# run and reports a smoothness it never demonstrated.
helper.reset_scroll()
restarts += 1
visible = helper.get_visible_portion()
column = int(helper.scroll_position)
if column == last_column:
duplicates += 1
last_column = column
if visible is None:
blanks += 1
else:
display.image.paste(visible, (0, 0))
# Every frame, not once before the loop. The scrolling state
# expires on its own inactivity threshold and takes the frame
# hold with it, so a scroll that announces itself once is
# presented at the wrong rate for all but its first moments --
# and its unchanged frames start taking the dirty-tracking
# skip, which returns without waiting for the panel at all.
# Every ticker re-announces per frame; so does this.
display.set_scrolling_state(True, frame_hold=choice.frame_hold)
display.update_display()
frames += 1
except KeyboardInterrupt:
print("\ninterrupted - reporting what was measured so far")
finally:
display.set_scrolling_state(False)
try:
display.clear()
except Exception as exc: # noqa: BLE001 - a lit panel is harmless; say so and go on
print(f"could not blank the panel: {exc}", file=sys.stderr)
if before is None:
print("interrupted during warm-up; nothing was graded", file=sys.stderr)
return 2
recorder.drain()
report = frame_soak.build_report(before, recorder.snapshot(), preview=False)
report["idle_refresh_hz"] = round(idle_hz, 2)
print()
frame_soak.print_report(report, args.max_late_pct)
held = report.get("held_refresh_hz")
if held:
drop = 100.0 * (idle_hz - held) / idle_hz
print(f"\npanel held ~{held:.1f}Hz while rendering, {drop:.1f}% below its "
f"{idle_hz:.1f}Hz idle rate (a widening gap is a render-cost "
"regression even with nothing late)")
if duplicates:
# A frame that shows the same columns as the one before it is work the
# panel did not need. It is not a miss -- the frame arrived on time --
# but it means the loop is presenting faster than the strip is moving.
print(f"duplicate {duplicates} frames advanced no pixels "
f"({100.0 * duplicates / max(1, frames):.2f}%)")
if blanks:
print(f"blank {blanks} frames had no visible slice to draw")
if restarts:
print(f"restarts {restarts} (the strip was scrolled through "
f"{restarts} time{'s' if restarts != 1 else ''})")
if args.json_path:
report.update({
"label": args.label or os.uname().nodename,
"bench": True,
"requested_pixels_per_second": requested,
"pixels_per_second": choice.pixels_per_second,
"pixels_per_frame": choice.pixels_per_frame,
"frame_hold": choice.frame_hold,
"busy_workers": args.busy,
"duplicate_frames": duplicates,
"blank_frames": blanks,
"strip_restarts": restarts,
"max_late_pct": args.max_late_pct,
"passed": frame_soak.passed(report, args.max_late_pct),
})
Path(args.json_path).write_text(json.dumps(report, indent=2) + "\n",
encoding="utf-8")
print(f"\nwrote {args.json_path}")
if report["late_pct"] is None:
return 2
return 0 if frame_soak.passed(report, args.max_late_pct) else 1
if __name__ == "__main__":
sys.exit(main())
+1
View File
@@ -299,5 +299,6 @@ def main():
if __name__ == '__main__':
import importlib.util
from typing import Optional
sys.exit(main())
+13 -6
View File
@@ -42,7 +42,7 @@ from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.common import frame_timing, scroll_config # noqa: E402
from src.common import scroll_config # noqa: E402
CONFIG = Path(__file__).resolve().parent.parent / "config" / "config.json"
@@ -99,13 +99,20 @@ def open_matrix(config, refresh_override=None):
def measure_refresh(config, seconds=6.0):
"""Actual refresh rate, by running uncapped and timing the swaps.
What an older Pi or a longer chain will really give you, as opposed to
whatever limit_refresh_rate_hz optimistically asks for. The timing loop
itself lives in src.common.frame_timing so the benchmark grades against
the same measurement this ladder is built from.
SwapOnVSync blocks until the panel's next refresh, so an unthrottled loop
runs at exactly the panel's rate. This is what an older Pi or a longer
chain will really give you, as opposed to whatever limit_refresh_rate_hz
optimistically asks for.
"""
matrix = open_matrix(config, refresh_override=0)
measured = frame_timing.measure_refresh_hz(matrix, seconds)
canvas = matrix.CreateFrameCanvas()
canvas = matrix.SwapOnVSync(canvas) # discard the first, it includes setup
frames = 0
started = time.perf_counter()
while time.perf_counter() - started < seconds:
canvas = matrix.SwapOnVSync(canvas)
frames += 1
measured = frames / (time.perf_counter() - started)
matrix.Clear()
return measured
-1
View File
@@ -9,7 +9,6 @@ This directory contains utility scripts for maintenance and system operations.
- **`wifi_monitor_daemon.py`** - Background daemon that monitors WiFi/Ethernet connection and manages access point mode
- **`pixlet_config_editor.sh`** - Opens Pixlet's own config UI for one installed Starlark app
- **`apply_dns_single_request.sh`** - Adds `options single-request` to the resolver (run by `ledmatrix-dns-fix.service`)
- **`auto_update_verify.py`** - Health check after an automatic update, rolling back if it fails (the updater copies it to `data/` before pulling and `ledmatrix-update-verify.service` runs that copy)
## Usage
+2 -31
View File
@@ -42,8 +42,6 @@ class WiFiMonitorDaemon:
"""
self.check_interval = check_interval
self.wifi_manager = WiFiManager()
# mtime of wifi_config.json as last loaded; see _reload_config_if_changed.
self._config_mtime = self._config_file_mtime()
self.running = True
self.last_state = None
# Counts consecutive checks where nmcli says "connected" but internet is unreachable.
@@ -59,32 +57,7 @@ class WiFiMonitorDaemon:
"""Handle shutdown signals"""
logger.info(f"Received signal {signum}, shutting down...")
self.running = False
def _config_file_mtime(self):
try:
return self.wifi_manager.config_path.stat().st_mtime_ns
except OSError:
return None
def _reload_config_if_changed(self):
"""Re-read wifi_config.json when it has changed on disk.
The web UI's auto-enable toggle (POST /api/v3/wifi/ap/auto-enable)
only writes the file; this process read it once at startup, so the
toggle did nothing until the daemon restarted. One stat per check.
"""
mtime = self._config_file_mtime()
if mtime is None or mtime == self._config_mtime:
return
before = self.wifi_manager.config.get("auto_enable_ap_mode", True)
self.wifi_manager._load_config()
# _load_config can itself save (it fills in missing keys), so take
# the mtime after it, or that save would trigger another reload.
self._config_mtime = self._config_file_mtime()
after = self.wifi_manager.config.get("auto_enable_ap_mode", True)
if after != before:
logger.info(f"wifi_config.json changed: auto_enable_ap_mode={after}")
def run(self):
"""Main daemon loop"""
logger.info("WiFi Monitor Daemon started")
@@ -105,8 +78,6 @@ class WiFiMonitorDaemon:
while self.running:
try:
self._reload_config_if_changed()
# One combined check that also returns the state it observed —
# the previous flow fetched status before AND after the check
# on top of the check's own internal fetch, each one several
@@ -248,7 +219,7 @@ def main():
parser.add_argument(
'--foreground',
action='store_true',
help='Accepted for compatibility; the daemon always runs in the foreground'
help='Run in foreground (for debugging)'
)
args = parser.parse_args()
+250
View File
@@ -0,0 +1,250 @@
#!/usr/bin/env python3
"""
Headless skin validator — render a skin against bundled fixture games at
multiple panel sizes without hardware, a network, or a running service.
python scripts/validate_skin.py --skin my-skin
python scripts/validate_skin.py --skin my-skin --sport baseball \
--size 128x32 --size 64x32 --output-dir /tmp/skin_renders
For each (mode x size) it checks: the manifest loads and its API version
matches, the render raises no exception, the canvas isn't blank, and the
render finishes inside a time budget (warn — the live renderer runs every
display-loop pass, and a Pi is far slower than your dev machine). PNGs are
saved (native plus 4x nearest-neighbor previews) so you can eyeball the
result. Exit code is non-zero when any check fails.
"""
import argparse
import json
import logging
import sys
import time
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(PROJECT_ROOT))
from PIL import Image, ImageDraw, ImageFont # noqa: E402
from src.common.font_layout import load_truetype # noqa: E402
FIXTURES_DIR = PROJECT_ROOT / "src" / "skin_system" / "fixtures"
MODES = ("live", "recent", "upcoming")
SPORTS = ("baseball", "basketball", "football", "hockey")
RENDER_BUDGET_S = 0.100
class FixtureHost:
"""Stands in for a SportsCore instance: fonts, logger, logo loading,
outlined text — everything build_context needs, no network."""
def __init__(self, sport: str, skin_options: dict) -> None:
self.sport = sport
self.sport_key = sport
self.skin_options = skin_options
self.logger = logging.getLogger(f"validate_skin.{sport}")
self.fonts = self._load_fonts()
self._logo_cache = {}
self.display_manager = None # build_context is always given a size
def _load_fonts(self) -> dict:
"""Load the SportsCore font set (TTF, with PIL default fallback)."""
fonts = {}
try:
press = str(PROJECT_ROOT / "assets/fonts/PressStart2P-Regular.ttf")
small = str(PROJECT_ROOT / "assets/fonts/4x6-font.ttf")
fonts['score'] = load_truetype(press, 10)
fonts['time'] = load_truetype(press, 8)
fonts['team'] = load_truetype(press, 8)
fonts['status'] = load_truetype(small, 6)
fonts['detail'] = load_truetype(small, 6)
fonts['rank'] = load_truetype(press, 10)
except IOError:
default = ImageFont.load_default()
for key in ('score', 'time', 'team', 'status', 'detail', 'rank'):
fonts[key] = default
return fonts
def _load_and_resize_logo(self, team_id: str, team_abbrev: str,
logo_path, logo_url) -> "Image.Image | None":
"""Load a fixture logo from disk (no downloads), cached per team."""
if team_abbrev in self._logo_cache:
return self._logo_cache[team_abbrev]
path = Path(logo_path)
if not path.is_absolute():
path = PROJECT_ROOT / path
if not path.exists():
return None
logo = Image.open(path).convert('RGBA')
self._logo_cache[team_abbrev] = logo
return logo
def _draw_text_with_outline(self, draw: "ImageDraw.ImageDraw", text: str,
position: tuple, font,
fill: tuple = (255, 255, 255),
outline_color: tuple = (0, 0, 0)) -> None:
"""Classic outlined scorebug text, same as SportsCore's helper."""
x, y = position
for dx, dy in [(-1, -1), (-1, 0), (-1, 1), (0, -1), (0, 1),
(1, -1), (1, 0), (1, 1)]:
draw.text((x + dx, y + dy), text, font=font, fill=outline_color)
draw.text((x, y), text, font=font, fill=fill)
def load_fixture(sport: str, mode: str) -> dict:
with open(FIXTURES_DIR / f"{sport}_{mode}.json", encoding="utf-8") as f:
game = json.load(f)
# Real view models carry start_time_utc as a UTC datetime, not a string.
if isinstance(game.get("start_time_utc"), str):
from datetime import datetime
game["start_time_utc"] = datetime.fromisoformat(game["start_time_utc"])
return game
def parse_size(value: str) -> "tuple[int, int]":
try:
w_text, h_text = value.lower().split("x")
w, h = int(w_text), int(h_text)
except ValueError as exc:
raise argparse.ArgumentTypeError(f"size must look like 128x32, got {value!r}") from exc
if w <= 0 or h <= 0:
raise argparse.ArgumentTypeError(f"size dimensions must be positive, got {value!r}")
return w, h
def parse_options(value: str) -> dict:
try:
options = json.loads(value)
except json.JSONDecodeError as exc:
raise argparse.ArgumentTypeError(f"options must be valid JSON: {exc.msg}") from exc
if not isinstance(options, dict):
raise argparse.ArgumentTypeError("options must be a JSON object")
return options
def display_path(path: Path) -> str:
"""Repo-relative when inside the repo, absolute otherwise (--output-dir
may point anywhere, e.g. /tmp/skin_renders)."""
try:
return str(path.relative_to(PROJECT_ROOT))
except ValueError:
return str(path)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--skin", required=True, help="skin id (directory name under skins/)")
parser.add_argument("--sport", choices=SPORTS,
help="fixture sport (default: first sport the skin targets, else baseball)")
parser.add_argument("--size", action="append", type=parse_size, dest="sizes",
metavar="WxH", help="panel size to render at (repeatable; default 128x32 and 64x32)")
parser.add_argument("--output-dir", type=Path,
default=PROJECT_ROOT / "skin_renders",
help="where rendered PNGs are written")
parser.add_argument("--options", type=parse_options, default={},
help="skin_options JSON to pass the skin")
args = parser.parse_args()
sizes = args.sizes or [(128, 32), (64, 32)]
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(name)s: %(message)s")
from src.skin_system import skin_runtime
from src.skin_system.skin_base import SKIN_API_VERSION
skins = skin_runtime.discover_skins()
manifest = skins.get(args.skin)
if manifest is None:
print(f"FAIL: skin '{args.skin}' not found under {skin_runtime.get_skins_directory()}")
if skins:
print(f" installed skins: {', '.join(sorted(skins))}")
return 1
sport = args.sport
if sport is None:
declared = skin_runtime.skin_targets(manifest)[0]
sport = next((s for s in declared if s in SPORTS), "baseball")
skin = skin_runtime.load_skin(args.skin, sport=sport, sport_key=sport,
options=args.options)
if skin is None:
print(f"FAIL: skin '{args.skin}' did not load "
f"(see log above; host API is {SKIN_API_VERSION})")
return 1
host = FixtureHost(sport, args.options)
args.output_dir.mkdir(parents=True, exist_ok=True)
failures = 0
rendered = 0
for mode in MODES:
game = load_fixture(sport, mode)
render = getattr(skin, f"render_{mode}")
for width, height in sizes:
label = f"{mode}@{width}x{height}"
try:
# Warm-up render absorbs one-time font/image loads, second
# render is the one timed against the budget.
ctx = skin_runtime.build_context(host, game, size=(width, height))
handled = render(ctx, dict(game))
if handled:
ctx = skin_runtime.build_context(host, game, size=(width, height))
started = time.monotonic()
handled = render(ctx, dict(game))
elapsed = time.monotonic() - started
else:
elapsed = 0.0
except Exception as e:
print(f"FAIL {label}: render raised {type(e).__name__}: {e}")
import traceback
traceback.print_exc()
failures += 1
continue
if not handled:
print(f"skip {label}: render_{mode} returned False (built-in renderer would be used)")
continue
if ctx.canvas.size != (width, height):
print(f"FAIL {label}: canvas was replaced/resized to {ctx.canvas.size} — draw onto ctx.canvas, never reassign it")
failures += 1
continue
if ctx.canvas.convert("L").getbbox() is None:
print(f"FAIL {label}: canvas is blank — render returned True but drew nothing")
failures += 1
continue
if elapsed > RENDER_BUDGET_S:
print(f"WARN {label}: render took {elapsed * 1000:.0f}ms "
f"(budget {RENDER_BUDGET_S * 1000:.0f}ms; a Pi is much slower than this machine)")
out = args.output_dir / f"{args.skin}_{sport}_{mode}_{width}x{height}.png"
ctx.canvas.save(out)
preview = ctx.canvas.resize((width * 4, height * 4), Image.NEAREST)
preview.save(out.with_name(out.stem + "_x4.png"))
print(f"ok {label}: {elapsed * 1000:.0f}ms -> {display_path(out)}")
rendered += 1
# Vegas card, once per mode at the first size (optional API)
try:
width, height = sizes[0]
ctx = skin_runtime.build_context(host, game, size=(width, height))
card = skin.render_vegas_card(ctx, dict(game))
if card is not None:
out = args.output_dir / f"{args.skin}_{sport}_{mode}_vegas.png"
card.save(out)
print(f"ok {mode} vegas card -> {display_path(out)}")
except Exception as e:
print(f"FAIL {mode} vegas card: {type(e).__name__}: {e}")
failures += 1
if rendered == 0 and failures == 0:
print(f"FAIL: skin '{args.skin}' rendered nothing — no render_<mode> returned True")
return 1
print(f"\n{'FAILED' if failures else 'PASSED'}: {rendered} renders, {failures} failures "
f"(PNGs in {args.output_dir})")
return 1 if failures else 0
if __name__ == "__main__":
sys.exit(main())
+28
View File
@@ -0,0 +1,28 @@
# skins/
> **Not supported yet.** The current scoreboard plugins don't render skins,
> so a skin placed here and selected in config has no effect, and the web UI
> and Plugin Store don't offer them. See
> [docs/SKIN_SYSTEM.md](../docs/SKIN_SYSTEM.md#status-not-supported-yet).
User-installable **visual skins** for the sports scoreboards. Each
subdirectory is one skin:
```text
skins/<skin-id>/
skin.json # manifest
skin.py # renderer (a ScoreboardSkin subclass)
preview.png # optional
```
- Install a skin: `git clone <skin repo> skins/<skin-id>`. The Plugin Store
refuses registry entries with `"type": "skin"` while skins don't render.
- Select it: set `"skin": "<skin-id>"` in the plugin's section of
`config/config.json`. The web UI no longer shows a Visual Skin dropdown.
- Build one: start from `example-classic-baseball/` and read
[docs/CREATING_SKINS.md](../docs/CREATING_SKINS.md). Validate with
`python scripts/validate_skin.py --skin <skin-id>`.
Skins survive plugin reinstalls/updates (that's why they live here and not in
the plugin's directory). A skin is Python at the same trust level as a
plugin — review before installing.
Binary file not shown.

After

Width:  |  Height:  |  Size: 5.3 KiB

+25
View File
@@ -0,0 +1,25 @@
{
"id": "example-classic-baseball",
"name": "Example: Classic Baseball",
"version": "1.0.0",
"author": "LEDMatrix",
"description": "Reference skin: a restyled baseball scorebug demonstrating the skin API. Copy this directory to start your own skin.",
"skin_api_version": "1.0.0",
"targets": {
"sports": [
"baseball"
],
"sport_keys": [
"mlb",
"milb"
]
},
"entry_point": "skin.py",
"class_name": "ClassicBaseballSkin",
"modes": [
"live",
"recent",
"upcoming"
],
"preview": "preview.png"
}
+131
View File
@@ -0,0 +1,131 @@
"""
Example: Classic Baseball — the reference skin.
Shows the whole skin API surface on purpose: adaptive regions
(scoreboard_regions), fitted text (ctx.layout.fit_text + ctx.draw_fit),
logos (ctx.load_logo + ctx.draw_image), raw PIL (ctx.draw for the bases
diamond), and per-user options (ctx.options). Everything is derived from
ctx and the game dict — a skin holds no state, does no I/O, and never
touches the display.
Copy this directory to skins/<your-skin-id>/, rename the class and the
manifest fields, and run:
python scripts/validate_skin.py --skin <your-skin-id>
"""
from src.adaptive_layout import LADDER_GRID, scoreboard_regions
from src.skin_system.skin_base import ScoreboardSkin, SkinContext
DEFAULT_ACCENT = (255, 200, 0)
class ClassicBaseballSkin(ScoreboardSkin):
"""Reference baseball skin: classic scorebug with bases/outs/count."""
def __init__(self, manifest: dict, options: dict):
super().__init__(manifest, options)
# Validate user options once at load time (fail fast, fall back
# gracefully) rather than surprising every render.
accent = self.options.get("accent_color", DEFAULT_ACCENT)
if (isinstance(accent, (list, tuple)) and len(accent) == 3
and all(isinstance(c, int) and 0 <= c <= 255 for c in accent)):
self._accent_color = tuple(accent)
else:
import logging
logging.getLogger(__name__).error(
"accent_color must be three 0-255 integers, got %r; using default", accent)
self._accent_color = DEFAULT_ACCENT
# -- shared pieces ----------------------------------------------------
def _accent(self, ctx: SkinContext) -> tuple:
"""Users can recolor the skin from config via skin_options."""
return self._accent_color
def _draw_card(self, ctx: SkinContext, game: dict, status: str,
center_lines: list, detail: str) -> None:
"""The common card: logos left/right, status on top, the given
center content, detail along the bottom."""
regions = scoreboard_regions(ctx.layout.bounds, ctx=ctx.layout)
ctx.draw_image(ctx.load_logo("away"), regions.away_slot,
cache_key=f"logo:{game.get('away_abbr')}")
ctx.draw_image(ctx.load_logo("home"), regions.home_slot,
cache_key=f"logo:{game.get('home_abbr')}")
if status:
fit = ctx.layout.fit_text(status, regions.status_band, LADDER_GRID)
ctx.draw_fit(fit, regions.status_band, color=self._accent(ctx))
if center_lines:
rows = regions.score_area.split_v(*[1] * len(center_lines))
for line, row in zip(center_lines, rows):
if line:
fit = ctx.layout.fit_text(line, row, LADDER_GRID)
ctx.draw_fit(fit, row)
if detail:
fit = ctx.layout.fit_text(detail, regions.detail_band, LADDER_GRID)
ctx.draw_fit(fit, regions.detail_band, color=(160, 160, 160))
def _draw_bases_and_outs(self, ctx: SkinContext, game: dict) -> None:
"""Raw-PIL escape hatch: a bases diamond + out dots in the bottom
band, sized from the layout scale so it works on any panel."""
size = ctx.layout.px(3, minimum=2) # half-diagonal of one base
gap = ctx.layout.px(1)
cx = ctx.width // 2
cy = ctx.height - (size * 2) - 1
bases = game.get("bases_occupied") or [False, False, False]
# (dx, dy) per base: first (right), second (top), third (left)
offsets = [(size + gap, 0), (0, -(size + gap)), (-(size + gap), 0)]
for occupied, (dx, dy) in zip(bases, offsets):
x, y = cx + dx, cy + dy
diamond = [(x, y - size), (x + size, y), (x, y + size), (x - size, y)]
if occupied:
ctx.draw.polygon(diamond, fill=self._accent(ctx))
else:
ctx.draw.polygon(diamond, outline=(110, 110, 110))
outs = min(int(game.get("outs") or 0), 3)
r = max(1, size - 1)
for i in range(3):
x = cx + (i - 1) * (2 * r + 2 * gap)
y = ctx.height - r - 1
dot = [x - r, y - r, x + r, y + r]
if i < outs:
ctx.draw.ellipse(dot, fill=(255, 255, 255))
else:
ctx.draw.ellipse(dot, outline=(110, 110, 110))
# -- the three modes --------------------------------------------------
def render_live(self, ctx: SkinContext, game: dict) -> bool:
half = "▲" if game.get("inning_half") == "top" else "▼"
inning = game.get("inning") or ""
status = f"{half}{inning}" if inning else game.get("status_text", "")
score = f"{game.get('away_score', '0')}-{game.get('home_score', '0')}"
count = f"{game.get('balls', 0)}-{game.get('strikes', 0)}"
self._draw_card(ctx, game, status, [score], "")
self._draw_bases_and_outs(ctx, game)
# Ball-strike count in the top-left corner, over the away logo.
fit = ctx.layout.fit_text(count, (ctx.width // 4, ctx.layout.px(8, minimum=6)), LADDER_GRID)
ctx.draw_fit(fit, ctx.layout.bounds.top_band(fit.height + 1).left_col(fit.width + 2),
color=(200, 200, 200))
return True
def render_recent(self, ctx: SkinContext, game: dict) -> bool:
score = f"{game.get('away_score', '0')}-{game.get('home_score', '0')}"
self._draw_card(ctx, game, game.get("status_text", "Final"),
[score], game.get("series_summary", ""))
return True
def render_upcoming(self, ctx: SkinContext, game: dict) -> bool:
matchup = f"{game.get('away_abbr', '')}@{game.get('home_abbr', '')}"
self._draw_card(ctx, game, game.get("game_date", ""),
[matchup, game.get("game_time", "")],
f"{game.get('away_record', '')} {game.get('home_record', '')}".strip())
return True
+1 -1
View File
@@ -4,5 +4,5 @@ LEDMatrix Display System
Core source package for the LED Matrix Display project.
"""
__version__ = "3.6.2"
__version__ = "3.5.0"
+7 -3
View File
@@ -29,9 +29,13 @@ from typing import Any, Optional, Tuple
from PIL import Image
# Re-exported by src.common for plugins, which import them from there.
RESAMPLE_LANCZOS = Image.Resampling.LANCZOS
RESAMPLE_NEAREST = Image.Resampling.NEAREST
# The one Pillow >= 9.1 compat shim (replaces the per-plugin copies).
try:
RESAMPLE_LANCZOS = Image.Resampling.LANCZOS
RESAMPLE_NEAREST = Image.Resampling.NEAREST
except AttributeError: # Pillow < 9.1
RESAMPLE_LANCZOS = Image.LANCZOS
RESAMPLE_NEAREST = Image.NEAREST
FIT_MODES = ("contain", "cover", "fill_height", "stretch")
+9 -25
View File
@@ -70,14 +70,7 @@ def _read(path):
def is_enabled(config):
# Only the {"enabled": true} object turns this on. A hand-edited
# non-dict (e.g. "auto_update": true) raised AttributeError here and
# aborted startup setup; the web UI's save replaces such a value with {}
# (disabled), so read it the same way.
section = config.get('auto_update')
if not isinstance(section, dict):
return False
return bool(section.get('enabled', False))
return bool((config.get('auto_update') or {}).get('enabled', False))
class UpdateHelperSetup:
@@ -208,26 +201,17 @@ class UpdateHelperSetup:
try:
self.result_file.parent.mkdir(parents=True, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=str(self.result_file.parent), prefix='.auto_update_setup_')
try:
with os.fdopen(fd, 'w', encoding='utf-8') as f:
json.dump(result, f, indent=2)
os.chmod(tmp, 0o644)
if self._web_ids and hasattr(os, 'chown'):
# Only root can give the file away; the result is readable
# (0644) either way, so a failed chown must not lose it.
try:
os.chown(tmp, *self._web_ids)
except OSError:
pass
os.replace(tmp, self.result_file)
except BaseException:
# Don't leave a .auto_update_setup_* file behind in the
# project dir every time the write fails.
with os.fdopen(fd, 'w', encoding='utf-8') as f:
json.dump(result, f, indent=2)
os.chmod(tmp, 0o644)
if self._web_ids and hasattr(os, 'chown'):
# Only root can give the file away; the result is readable
# (0644) either way, so a failed chown must not lose it.
try:
os.unlink(tmp)
os.chown(tmp, *self._web_ids)
except OSError:
pass
raise
os.replace(tmp, self.result_file)
except OSError as e:
logger.warning("Could not record automatic update setup result: %s", e)
return result
+73 -71
View File
@@ -26,7 +26,6 @@ from enum import Enum
from concurrent.futures import ThreadPoolExecutor
import pytz
from src.cache_manager import CacheManager
from src.common.json_body import response_json
from src.common.espn_dates import (
RANGE_RETRY_SECONDS,
_note_range_rejected,
@@ -103,32 +102,6 @@ class FetchResult:
# FAILED, which turns "you cancelled this" into "this errored".
final_status: Optional[FetchStatus] = None
class _ConnectionRetryingSession:
"""``session.get`` that retries a connection error a few times.
For ESPN date chunks, which bypass _make_request_with_retry: a failed
chunk is logged and skipped, so a brief network blip would otherwise drop
a month from a cached season. That protection used to come from the
session adapter's own retries, which every other request stacked with the
retry loop.
"""
ATTEMPTS = 3
DELAY = 0.5
def __init__(self, session):
self._session = session
def get(self, *args, **kwargs):
for attempt in range(self.ATTEMPTS):
try:
return self._session.get(*args, **kwargs)
except requests.ConnectionError:
if attempt == self.ATTEMPTS - 1:
raise
time.sleep(self.DELAY * (attempt + 1))
class BackgroundDataService:
"""
Background data service for fetching season data without blocking the main thread.
@@ -157,14 +130,19 @@ class BackgroundDataService:
# Thread management
self.executor = ThreadPoolExecutor(max_workers=max_workers, thread_name_prefix="BackgroundData")
# cache_key -> request_id for fetches currently in flight, so a second
# submit for the same key joins the running fetch instead of starting
# another. It is the normal case: a sport's Recent and Upcoming
# managers miss the cache for the same season schedule together.
# cache_key -> request_id for fetches currently in flight. Submitting
# the same key twice used to start two identical fetches: request_id
# carries a millisecond timestamp, so every submit looked new, and
# active_requests is keyed by it rather than by what is being fetched.
# On a real board the season-schedule key is requested by both the
# Recent and the Upcoming manager, which miss the cache in the same
# millisecond and each download and parse the same payload.
self._inflight_by_cache_key: Dict[str, str] = {}
# Makes every request_id unique. The id also carries a millisecond
# timestamp, but two submits can share a millisecond, and a joiner
# uses the id as its handle for get_result().
# request_id was sport_year_milliseconds, which is not unique: two
# submits inside the same millisecond produced the SAME id, so one
# silently replaced the other in active_requests and completed_requests.
# Rare before, but dedupe hands this id back to every joiner as their
# handle for get_result(), so it has to be unique. A counter is enough.
self._request_seq = itertools.count()
self.active_requests: Dict[str, FetchRequest] = {}
self.completed_requests: Dict[str, FetchResult] = {}
@@ -189,16 +167,10 @@ class BackgroundDataService:
'average_fetch_time': 0.0
}
# Session for HTTP requests. No retries at the adapter: a fetch goes
# through _make_request_with_retry (max_retries + 1 attempts with
# exponential backoff, logged), and date-range chunks through
# _ConnectionRetryingSession. With the adapter also retrying
# connection errors three times, a dead network cost up to 16
# connection attempts per request and held one of the few worker
# threads for all of them.
# Session for HTTP requests
self.session = requests.Session()
self.session.mount('http://', requests.adapters.HTTPAdapter(max_retries=0))
self.session.mount('https://', requests.adapters.HTTPAdapter(max_retries=0))
self.session.mount('http://', requests.adapters.HTTPAdapter(max_retries=3))
self.session.mount('https://', requests.adapters.HTTPAdapter(max_retries=3))
# Default headers: core's shared set (real User-Agent, no hand-set
# Accept-Encoding) -- see src/common/api_helper.py.
@@ -213,9 +185,9 @@ class BackgroundDataService:
This ensures Recent/Upcoming managers and background service
use the same cache keys.
"""
# Same format as CacheManager.generate_sport_cache_key(), built here
# rather than by constructing a CacheManager (config load, cache-dir
# probing) on every submit without a cache_key.
# Same format as CacheManager.generate_sport_cache_key(). This used to
# build a whole CacheManager to call it -- config load, cache-dir
# probing with test writes -- on every submit without a cache_key.
if date_str is None:
date_str = datetime.now(pytz.utc).strftime('%Y%m%d')
return f"{sport}_{date_str}"
@@ -279,19 +251,15 @@ class BackgroundDataService:
# same object the dict holds.
self.completed_requests[request_id] = result
# The callback runs outside the lock, as on the worker path: it is
# plugin code, and holding the service lock through it blocked
# every worker's result bookkeeping (and any other thread's
# submit) for as long as the callback took.
if callback:
try:
callback(result)
except Exception as e:
logger.error(f"Error in callback for request {request_id}: {e}")
self._release_payload(result)
if callback:
try:
callback(result)
except Exception as e:
logger.error(f"Error in callback for request {request_id}: {e}")
self._release_payload(result)
logger.debug(f"Cache hit for {sport} {year} data")
return request_id
logger.debug(f"Cache hit for {sport} {year} data")
return request_id
# limit above 500 makes an ESPN *scoreboard* return a truncated list
# (src/common/espn_dates.py). Other endpoints need more: /teams has 762
@@ -362,8 +330,10 @@ class BackgroundDataService:
try:
with self._lock:
# A request cancelled while it sat in the executor queue stays
# cancelled: no download, no cache write, no callback.
# A request cancelled while it sat in the executor queue must
# stay cancelled. Overwriting the status here undid the cancel
# outright: the worker went on to download, cache and call back
# for work the caller had already withdrawn.
if request.status == FetchStatus.CANCELLED:
cancelled_before_start = True
else:
@@ -419,7 +389,7 @@ class BackgroundDataService:
response.raise_for_status()
else:
response.raise_for_status()
data = response_json(response)
data = response.json()
# Validate data structure
if not isinstance(data, dict):
@@ -492,9 +462,10 @@ class BackgroundDataService:
logger.error(f"Failed to fetch {request.sport} {request.year} data: {error_msg}")
with self._lock:
# A cancelled request stays CANCELLED even when its fetch
# failed: the finally block skips callbacks only for
# CANCELLED, and nobody is waiting on this fetch any more.
# Don't relabel a cancelled request. The callback gate in the
# finally block only suppresses CANCELLED, so promoting it to
# FAILED here delivered an error callback for a fetch nobody
# was waiting on any more.
if request.status != FetchStatus.CANCELLED:
request.status = FetchStatus.FAILED
request.error = error_msg
@@ -554,13 +525,20 @@ class BackgroundDataService:
except Exception as e:
logger.error(f"Error in callback for request {request.id}: {e}")
# Released after the loop, never inside it: every callback holds
# the same FetchResult (a sport's recent, upcoming and live
# managers usually share one fetch), so a release between
# deliveries would hand the later ones `result.data is None`.
# Released AFTER the loop, not inside it. Every callback here holds
# the same FetchResult, so releasing per-delivery handed the first
# one the data and every joiner `result.data is None` -- which is
# not a quiet degradation: they read `result.data.get('events')` and
# raise AttributeError, which this very loop catches and logs, so
# the symptom was one ERROR line and a manager that silently never
# got its schedule. Deduplication is the normal case, not a corner:
# a sport's recent, upcoming and live managers all ride one season
# fetch.
#
# Only when there were callbacks: a request submitted without one
# collects its payload by polling get_result().
# Guarded on `callbacks`, because a request submitted without one
# has no other way to collect its payload than polling get_result().
# The old per-delivery release got that right by accident: an empty
# list never entered the loop body.
if callbacks:
self._release_payload(result)
request.result = None
@@ -596,7 +574,7 @@ class BackgroundDataService:
"""
logger.info("Recovering %s %s from a rejected date range", request.sport, request.year)
return fetch_espn_date_chunks(
_ConnectionRetryingSession(self.session),
self.session,
request.url,
params=request.params,
headers=request.headers,
@@ -742,6 +720,9 @@ class BackgroundDataService:
'completed_requests_count': len(self.completed_requests),
'max_completed_requests': self._max_completed_requests,
'completed_requests_usage_percent': (len(self.completed_requests) / self._max_completed_requests * 100) if self._max_completed_requests > 0 else 0,
# Nothing is queued outside the executor; kept for callers
# that read the key.
'queue_size': 0,
'last_cleanup': self._last_completed_requests_cleanup,
'cleanup_interval': self._completed_requests_cleanup_interval
}
@@ -811,6 +792,27 @@ class BackgroundDataService:
return removed_count
def clear_completed_requests(self, older_than_hours: int = 24):
"""
Clear completed requests older than specified time.
Args:
older_than_hours: Clear requests older than this many hours
"""
cutoff_time = time.time() - (older_than_hours * 3600)
with self._lock:
to_remove = []
for request_id, result in self.completed_requests.items():
if result.completed_at < cutoff_time:
to_remove.append(request_id)
for request_id in to_remove:
del self.completed_requests[request_id]
if to_remove:
logger.info(f"Cleared {len(to_remove)} old completed requests")
def shutdown(self, wait: bool = True):
"""
Shutdown the background data service.
+110 -111
View File
@@ -83,31 +83,14 @@ BUNDLED_FONTS: frozenset[str] = frozenset({
_CONFIG_REL = Path("config/config.json")
_SECRETS_REL = Path("config/config_secrets.json")
_WIFI_REL = Path("config/wifi_config.json")
# A YouTube Music session: pure user state that has to be re-authenticated by
# hand if lost, so a restore must bring it back.
# Sits in config/ next to the three above and is pure user state — a
# YouTube Music session that has to be re-authenticated by hand if lost.
# It was omitted from backups, so a restore silently signed the user out.
_YTM_REL = Path("config/ytm_auth.json")
_FONTS_REL = Path("assets/fonts")
_PLUGIN_UPLOADS_REL = Path("assets/plugins")
_STATE_REL = Path("data/plugin_state.json")
#: The sections that are one file each: (section name, path, the
#: RestoreOptions flag that restores it). create, preview, validate and
#: restore all walk this table. ytm_auth follows restore_wifi: it is
#: device-local auth like the Wi-Fi settings, and a toggle of its own for one
#: file would be noise in the restore dialog.
_SINGLE_FILE_SECTIONS: Tuple[Tuple[str, Path, str], ...] = (
("config", _CONFIG_REL, "restore_config"),
("secrets", _SECRETS_REL, "restore_secrets"),
("wifi", _WIFI_REL, "restore_wifi"),
("ytm_auth", _YTM_REL, "restore_wifi"),
)
#: Sections holding credentials. Restored onto a device that has no copy yet,
#: they would otherwise take the extracted temp file's umask mode (0o644,
#: world-readable); 0o640 matches what config_manager_atomic gives secrets.
_PRIVATE_SECTION_RELS = frozenset({_SECRETS_REL, _WIFI_REL, _YTM_REL})
_PRIVATE_FILE_MODE = 0o640
MANIFEST_NAME = "manifest.json"
PLUGINS_MANIFEST_NAME = "plugins.json"
@@ -157,18 +140,34 @@ class RestoreResult:
# ---------------------------------------------------------------------------
def _ledmatrix_version() -> str:
"""The release of the running core (``src.__version__``), recorded in the
manifest so a restore can tell which release wrote the backup."""
from src import __version__
return __version__
def _ledmatrix_version(project_root: Path) -> str:
"""Best-effort version string for the current install."""
version_file = project_root / "VERSION"
if version_file.exists():
try:
return version_file.read_text(encoding="utf-8").strip() or "unknown"
except OSError:
pass
head_file = project_root / ".git" / "HEAD"
if head_file.exists():
try:
head = head_file.read_text(encoding="utf-8").strip()
if head.startswith("ref: "):
ref = head[5:]
ref_path = project_root / ".git" / ref
if ref_path.exists():
return ref_path.read_text(encoding="utf-8").strip()[:12] or "unknown"
return head[:12] or "unknown"
except OSError:
pass
return "unknown"
def _build_manifest(contents: List[str]) -> Dict[str, Any]:
def _build_manifest(contents: List[str], project_root: Path) -> Dict[str, Any]:
return {
"schema_version": SCHEMA_VERSION,
"created_at": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
"ledmatrix_version": _ledmatrix_version(),
"ledmatrix_version": _ledmatrix_version(project_root),
"hostname": socket.gethostname(),
"contents": contents,
}
@@ -179,34 +178,13 @@ def _build_manifest(contents: List[str]) -> Dict[str, Any]:
# ---------------------------------------------------------------------------
def _plugins_directory(project_root: Path) -> Path:
"""The plugin install directory: ``plugin_system.plugins_directory`` from
config/config.json (relative to ``project_root`` unless absolute), or
``plugin-repos`` when the config does not say or cannot be read."""
configured: Any = None
try:
with (project_root / _CONFIG_REL).open("r", encoding="utf-8") as f:
config = json.load(f)
if isinstance(config, dict):
plugin_system = config.get("plugin_system")
if isinstance(plugin_system, dict):
configured = plugin_system.get("plugins_directory")
except (OSError, json.JSONDecodeError):
pass
if not isinstance(configured, str) or not configured.strip():
configured = "plugin-repos"
path = Path(configured)
return path if path.is_absolute() else project_root / path
def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
"""
Return a list of currently-installed plugins suitable for the backup
manifest. Each entry has ``plugin_id`` and ``version``.
Reads ``data/plugin_state.json`` if present, then adds any plugin it
does not list from the ``manifest.json`` files in the configured plugin
directory (see :func:`_plugins_directory`).
Reads ``data/plugin_state.json`` if present; otherwise walks the plugin
directory and reads each ``manifest.json``.
"""
plugins: Dict[str, Dict[str, Any]] = {}
@@ -228,7 +206,8 @@ def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
except (OSError, json.JSONDecodeError) as e:
logger.warning("Could not read plugin_state.json: %s", e)
plugins_root = _plugins_directory(project_root)
# Fall back to scanning plugin-repos/ for manifests.
plugins_root = project_root / "plugin-repos"
if plugins_root.exists():
for entry in sorted(plugins_root.iterdir()):
if not entry.is_dir():
@@ -241,10 +220,6 @@ def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
data = json.load(f)
except (OSError, json.JSONDecodeError):
continue
# Valid JSON that is not an object (a list, a bare string) would
# raise AttributeError on .get() and abort the whole export.
if not isinstance(data, dict):
continue
plugin_id = data.get("id") or entry.name
if plugin_id not in plugins:
plugins[plugin_id] = {
@@ -320,17 +295,22 @@ def create_backup(
contents: List[str] = []
# Stream directly to a temp file so we never hold the whole ZIP in memory.
# The name is unique per call: a fixed "<zip>.tmp" was shared by two
# exports started in the same second, which then wrote the same file.
fd, tmp_name = tempfile.mkstemp(dir=str(output_dir), prefix=f".{zip_name}.", suffix=".tmp")
os.close(fd)
tmp_path = Path(tmp_name)
tmp_path = zip_path.with_suffix(".zip.tmp")
try:
with zipfile.ZipFile(tmp_path, "w", compression=zipfile.ZIP_DEFLATED) as zf:
for section, rel, _flag in _SINGLE_FILE_SECTIONS:
if (project_root / rel).exists():
zf.write(project_root / rel, rel.as_posix())
contents.append(section)
# Config files.
if (project_root / _CONFIG_REL).exists():
zf.write(project_root / _CONFIG_REL, _CONFIG_REL.as_posix())
contents.append("config")
if (project_root / _SECRETS_REL).exists():
zf.write(project_root / _SECRETS_REL, _SECRETS_REL.as_posix())
contents.append("secrets")
if (project_root / _WIFI_REL).exists():
zf.write(project_root / _WIFI_REL, _WIFI_REL.as_posix())
contents.append("wifi")
if (project_root / _YTM_REL).exists():
zf.write(project_root / _YTM_REL, _YTM_REL.as_posix())
contents.append("ytm_auth")
# User-uploaded fonts.
user_fonts = iter_user_fonts(project_root)
@@ -358,27 +338,10 @@ def create_backup(
contents.append("plugins")
# Manifest goes last so that `contents` reflects what we actually wrote.
manifest = _build_manifest(contents)
manifest = _build_manifest(contents, project_root)
zf.writestr(MANIFEST_NAME, json.dumps(manifest, indent=2))
# Same-second exports share a timestamp; number the later one rather
# than replacing the backup the first one just returned. The name is
# claimed with an exclusive create (O_EXCL fails if it exists), so two
# exports finishing together can't both pick the same free name; the
# replace then swaps the finished archive in over our own placeholder.
suffix = 2
while True:
try:
os.close(os.open(zip_path, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o600))
break
except FileExistsError:
zip_path = output_dir / f"{Path(zip_name).stem}-{suffix}.zip"
suffix += 1
try:
os.replace(tmp_path, zip_path)
except BaseException:
zip_path.unlink(missing_ok=True)
raise
os.replace(tmp_path, zip_path)
except Exception:
tmp_path.unlink(missing_ok=True)
raise
@@ -389,16 +352,15 @@ def create_backup(
def preview_backup_contents(project_root: Path) -> Dict[str, Any]:
"""Return a summary of what ``create_backup`` would include."""
project_root = Path(project_root).resolve()
preview: Dict[str, Any] = {
f"has_{section}": (project_root / rel).exists()
for section, rel, _flag in _SINGLE_FILE_SECTIONS
}
preview.update({
return {
"has_config": (project_root / _CONFIG_REL).exists(),
"has_secrets": (project_root / _SECRETS_REL).exists(),
"has_wifi": (project_root / _WIFI_REL).exists(),
"has_ytm_auth": (project_root / _YTM_REL).exists(),
"user_fonts": [p.name for p in iter_user_fonts(project_root)],
"plugin_uploads": len(iter_plugin_uploads(project_root)),
"plugins": list_installed_plugins(project_root),
})
return preview
}
# ---------------------------------------------------------------------------
@@ -469,10 +431,15 @@ def validate_backup(zip_path: Path) -> Tuple[bool, str, Dict[str, Any]]:
{},
)
detected: List[str] = [
section for section, rel, _flag in _SINGLE_FILE_SECTIONS
if rel.as_posix() in names
]
detected: List[str] = []
if _CONFIG_REL.as_posix() in names:
detected.append("config")
if _SECRETS_REL.as_posix() in names:
detected.append("secrets")
if _WIFI_REL.as_posix() in names:
detected.append("wifi")
if _YTM_REL.as_posix() in names:
detected.append("ytm_auth")
if any(n.startswith(_FONTS_REL.as_posix() + "/") for n in names):
detected.append("fonts")
if any(
@@ -481,8 +448,7 @@ def validate_backup(zip_path: Path) -> Tuple[bool, str, Dict[str, Any]]:
):
detected.append("plugin_uploads")
# Whatever the archive's manifest holds; checked below.
plugins: Any = []
plugins: List[Dict[str, Any]] = []
if PLUGINS_MANIFEST_NAME in names:
try:
plugins = json.loads(zf.read(PLUGINS_MANIFEST_NAME).decode("utf-8"))
@@ -525,7 +491,7 @@ def _extract_zip_safe(zip_path: Path, dest_dir: Path) -> None:
shutil.copyfileobj(src, dst, length=64 * 1024)
def _copy_file(src: Path, dst: Path, new_mode: Optional[int] = None) -> None:
def _copy_file(src: Path, dst: Path) -> None:
"""Replace ``dst`` with ``src``, atomically, without needing to own ``dst``.
``shutil.copy2`` opens the destination for writing, so it needs write
@@ -541,7 +507,6 @@ def _copy_file(src: Path, dst: Path, new_mode: Optional[int] = None) -> None:
The destination's existing mode is preserved when there is one, so
restoring secrets does not silently widen them to the umask default.
When there is none, ``new_mode`` (if given) is used instead of ``src``'s.
"""
dst.parent.mkdir(parents=True, exist_ok=True)
@@ -563,8 +528,6 @@ def _copy_file(src: Path, dst: Path, new_mode: Optional[int] = None) -> None:
shutil.copyfile(src, tmp_path)
if existing_mode is not None:
os.chmod(tmp_path, existing_mode)
elif new_mode is not None:
os.chmod(tmp_path, new_mode)
else:
shutil.copymode(src, tmp_path)
if existing_owner is not None and hasattr(os, 'chown'):
@@ -621,19 +584,55 @@ def restore_backup(
result.errors.append("Failed to extract backup")
return result
for section, rel, flag in _SINGLE_FILE_SECTIONS:
if not (tmp_dir / rel).exists():
continue
if not getattr(options, flag):
result.skipped.append(section)
continue
# Main config.
if options.restore_config and (tmp_dir / _CONFIG_REL).exists():
try:
_copy_file(tmp_dir / rel, project_root / rel,
new_mode=_PRIVATE_FILE_MODE if rel in _PRIVATE_SECTION_RELS else None)
result.restored.append(section)
_copy_file(tmp_dir / _CONFIG_REL, project_root / _CONFIG_REL)
result.restored.append("config")
except OSError as e:
logger.error("[Backup] Failed to restore %s: %s", rel.name, e, exc_info=True)
result.errors.append(f"Failed to restore {rel.name}")
logger.error("[Backup] Failed to restore config.json: %s", e, exc_info=True)
result.errors.append("Failed to restore config.json")
elif (tmp_dir / _CONFIG_REL).exists():
result.skipped.append("config")
# Secrets.
if options.restore_secrets and (tmp_dir / _SECRETS_REL).exists():
try:
_copy_file(tmp_dir / _SECRETS_REL, project_root / _SECRETS_REL)
result.restored.append("secrets")
except OSError as e:
logger.error(
"[Backup] Failed to restore config_secrets.json: %s", e, exc_info=True
)
result.errors.append("Failed to restore config_secrets.json")
elif (tmp_dir / _SECRETS_REL).exists():
result.skipped.append("secrets")
# WiFi.
if options.restore_wifi and (tmp_dir / _WIFI_REL).exists():
try:
_copy_file(tmp_dir / _WIFI_REL, project_root / _WIFI_REL)
result.restored.append("wifi")
except OSError as e:
logger.error(
"[Backup] Failed to restore wifi_config.json: %s", e, exc_info=True
)
result.errors.append("Failed to restore wifi_config.json")
elif (tmp_dir / _WIFI_REL).exists():
result.skipped.append("wifi")
# YouTube Music session. Follows restore_wifi rather than getting its
# own flag: it is device-local auth in the same sense, and a separate
# toggle for one file would be noise in the restore dialog.
if options.restore_wifi and (tmp_dir / _YTM_REL).exists():
try:
_copy_file(tmp_dir / _YTM_REL, project_root / _YTM_REL)
result.restored.append("ytm_auth")
except OSError as e:
logger.error("[Backup] Failed to restore ytm_auth.json: %s", e, exc_info=True)
result.errors.append("Failed to restore ytm_auth.json")
elif (tmp_dir / _YTM_REL).exists():
result.skipped.append("ytm_auth")
# User fonts — skip anything that collides with a bundled font.
tmp_fonts = tmp_dir / _FONTS_REL
+371
View File
@@ -0,0 +1,371 @@
"""
Abstract API Data Extraction Layer
This module provides a pluggable system for extracting game data from different
sports APIs. Each sport can have its own extractor that handles sport-specific
fields and data structures.
"""
from abc import ABC, abstractmethod
from typing import Dict, Optional
import logging
from datetime import datetime
import pytz
class APIDataExtractor(ABC):
"""Abstract base class for API data extraction."""
def __init__(self, logger: logging.Logger):
self.logger = logger
@abstractmethod
def extract_game_details(self, game_event: Dict) -> Optional[Dict]:
"""Extract common game details from raw API data."""
@abstractmethod
def get_sport_specific_fields(self, game_event: Dict) -> Dict:
"""Extract sport-specific fields (downs, innings, periods, etc.)."""
def _extract_common_details(self, game_event: Dict) -> tuple[Dict | None, Dict | None, Dict | None, Dict | None, Dict | None]:
"""Extract common game details that work across all sports."""
if not game_event:
return None, None, None, None, None
try:
competition = game_event["competitions"][0]
status = competition["status"]
competitors = competition["competitors"]
game_date_str = game_event["date"]
situation = competition.get("situation")
# Parse game time
start_time_utc = None
try:
# Parse the datetime string
if game_date_str.endswith('Z'):
game_date_str = game_date_str.replace('Z', '+00:00')
dt = datetime.fromisoformat(game_date_str)
# Ensure the datetime is UTC-aware (fromisoformat may create timezone-aware but not pytz.UTC)
if dt.tzinfo is None:
# If naive, assume it's UTC
start_time_utc = dt.replace(tzinfo=pytz.UTC)
else:
# Convert to pytz.UTC for consistency
start_time_utc = dt.astimezone(pytz.UTC)
except ValueError:
self.logger.warning(f"Could not parse game date: {game_date_str}")
# Extract teams
home_team = next((c for c in competitors if c.get("homeAway") == "home"), None)
away_team = next((c for c in competitors if c.get("homeAway") == "away"), None)
if not home_team or not away_team:
self.logger.warning(f"Could not find home or away team in event: {game_event.get('id')}")
return None, None, None, None, None
return {
"game_event": game_event,
"competition": competition,
"status": status,
"situation": situation,
"start_time_utc": start_time_utc,
"home_team": home_team,
"away_team": away_team
}, home_team, away_team, status, situation
except Exception as e:
self.logger.error(f"Error extracting common details: {e}")
return None, None, None, None, None
class ESPNFootballExtractor(APIDataExtractor):
"""ESPN API extractor for football (NFL/NCAA)."""
def extract_game_details(self, game_event: Dict) -> Optional[Dict]:
"""Extract football game details from ESPN API."""
common_data, home_team, away_team, status, situation = self._extract_common_details(game_event)
if not common_data:
return None
try:
# Extract basic team info
home_abbr = home_team["team"]["abbreviation"]
away_abbr = away_team["team"]["abbreviation"]
home_score = home_team.get("score", "0")
away_score = away_team.get("score", "0")
# Extract sport-specific fields
sport_fields = self.get_sport_specific_fields(game_event)
# Build game details
details = {
"id": game_event.get("id"),
"home_abbr": home_abbr,
"away_abbr": away_abbr,
"home_score": str(home_score),
"away_score": str(away_score),
"home_team_name": home_team["team"].get("displayName", ""),
"away_team_name": away_team["team"].get("displayName", ""),
"status_text": status["type"].get("shortDetail", ""),
"is_live": status["type"]["state"] == "in",
"is_final": status["type"]["state"] == "post",
"is_upcoming": status["type"]["state"] == "pre",
**sport_fields # Add sport-specific fields
}
return details
except Exception as e:
self.logger.error(f"Error extracting football game details: {e}")
return None
def get_sport_specific_fields(self, game_event: Dict) -> Dict:
"""Extract football-specific fields."""
try:
competition = game_event["competitions"][0]
status = competition["status"]
situation = competition.get("situation", {})
sport_fields = {
"down": "",
"distance": "",
"possession": "",
"is_redzone": False,
"home_timeouts": 0,
"away_timeouts": 0,
"scoring_event": ""
}
if situation and status["type"]["state"] == "in":
sport_fields.update({
"down": situation.get("down", ""),
"distance": situation.get("distance", ""),
"possession": situation.get("possession", ""),
"is_redzone": situation.get("isRedZone", False),
"home_timeouts": situation.get("homeTimeouts", 0),
"away_timeouts": situation.get("awayTimeouts", 0)
})
# Detect scoring events
status_detail = status["type"].get("detail", "").lower()
if "touchdown" in status_detail or "field goal" in status_detail:
sport_fields["scoring_event"] = status_detail
return sport_fields
except Exception as e:
self.logger.error(f"Error extracting football-specific fields: {e}")
return {}
class ESPNBaseballExtractor(APIDataExtractor):
"""ESPN API extractor for baseball (MLB)."""
def extract_game_details(self, game_event: Dict) -> Optional[Dict]:
"""Extract baseball game details from ESPN API."""
common_data, home_team, away_team, status, situation = self._extract_common_details(game_event)
if not common_data:
return None
try:
# Extract basic team info
home_abbr = home_team["team"]["abbreviation"]
away_abbr = away_team["team"]["abbreviation"]
home_score = home_team.get("score", "0")
away_score = away_team.get("score", "0")
# Extract sport-specific fields
sport_fields = self.get_sport_specific_fields(game_event)
# Build game details
details = {
"id": game_event.get("id"),
"home_abbr": home_abbr,
"away_abbr": away_abbr,
"home_score": str(home_score),
"away_score": str(away_score),
"home_team_name": home_team["team"].get("displayName", ""),
"away_team_name": away_team["team"].get("displayName", ""),
"status_text": status["type"].get("shortDetail", ""),
"is_live": status["type"]["state"] == "in",
"is_final": status["type"]["state"] == "post",
"is_upcoming": status["type"]["state"] == "pre",
**sport_fields # Add sport-specific fields
}
return details
except Exception as e:
self.logger.error(f"Error extracting baseball game details: {e}")
return None
def get_sport_specific_fields(self, game_event: Dict) -> Dict:
"""Extract baseball-specific fields."""
try:
competition = game_event["competitions"][0]
status = competition["status"]
situation = competition.get("situation", {})
sport_fields = {
"inning": "",
"outs": 0,
"bases": "",
"strikes": 0,
"balls": 0,
"pitcher": "",
"batter": ""
}
if situation and status["type"]["state"] == "in":
sport_fields.update({
"inning": situation.get("inning", ""),
"outs": situation.get("outs", 0),
"bases": situation.get("bases", ""),
"strikes": situation.get("strikes", 0),
"balls": situation.get("balls", 0),
"pitcher": situation.get("pitcher", ""),
"batter": situation.get("batter", "")
})
return sport_fields
except Exception as e:
self.logger.error(f"Error extracting baseball-specific fields: {e}")
return {}
class ESPNHockeyExtractor(APIDataExtractor):
"""ESPN API extractor for hockey (NHL/NCAA)."""
def extract_game_details(self, game_event: Dict) -> Optional[Dict]:
"""Extract hockey game details from ESPN API."""
common_data, home_team, away_team, status, situation = self._extract_common_details(game_event)
if not common_data:
return None
try:
# Extract basic team info
home_abbr = home_team["team"]["abbreviation"]
away_abbr = away_team["team"]["abbreviation"]
home_score = home_team.get("score", "0")
away_score = away_team.get("score", "0")
# Extract sport-specific fields
sport_fields = self.get_sport_specific_fields(game_event)
# Build game details
details = {
"id": game_event.get("id"),
"home_abbr": home_abbr,
"away_abbr": away_abbr,
"home_score": str(home_score),
"away_score": str(away_score),
"home_team_name": home_team["team"].get("displayName", ""),
"away_team_name": away_team["team"].get("displayName", ""),
"status_text": status["type"].get("shortDetail", ""),
"is_live": status["type"]["state"] == "in",
"is_final": status["type"]["state"] == "post",
"is_upcoming": status["type"]["state"] == "pre",
**sport_fields # Add sport-specific fields
}
return details
except Exception as e:
self.logger.error(f"Error extracting hockey game details: {e}")
return None
def get_sport_specific_fields(self, game_event: Dict) -> Dict:
"""Extract hockey-specific fields."""
try:
competition = game_event["competitions"][0]
status = competition["status"]
situation = competition.get("situation", {})
sport_fields = {
"period": "",
"period_text": "",
"power_play": False,
"penalties": "",
"shots_on_goal": {"home": 0, "away": 0}
}
if situation and status["type"]["state"] == "in":
period = status.get("period", 0)
period_text = ""
if period == 1:
period_text = "P1"
elif period == 2:
period_text = "P2"
elif period == 3:
period_text = "P3"
elif period > 3:
period_text = f"OT{period-3}"
sport_fields.update({
"period": str(period),
"period_text": period_text,
"power_play": situation.get("isPowerPlay", False),
"penalties": situation.get("penalties", ""),
"shots_on_goal": {
"home": situation.get("homeShots", 0),
"away": situation.get("awayShots", 0)
}
})
return sport_fields
except Exception as e:
self.logger.error(f"Error extracting hockey-specific fields: {e}")
return {}
class SoccerAPIExtractor(APIDataExtractor):
"""Generic extractor for soccer APIs (different structure than ESPN)."""
def extract_game_details(self, game_event: Dict) -> Optional[Dict]:
"""Extract soccer game details from various soccer APIs."""
# This would need to be adapted based on the specific soccer API being used
# For now, return a basic structure
try:
return {
"id": game_event.get("id"),
"home_abbr": game_event.get("home_team", {}).get("abbreviation", ""),
"away_abbr": game_event.get("away_team", {}).get("abbreviation", ""),
"home_score": str(game_event.get("home_score", "0")),
"away_score": str(game_event.get("away_score", "0")),
"home_team_name": game_event.get("home_team", {}).get("name", ""),
"away_team_name": game_event.get("away_team", {}).get("name", ""),
"status_text": game_event.get("status", ""),
"is_live": game_event.get("is_live", False),
"is_final": game_event.get("is_final", False),
"is_upcoming": game_event.get("is_upcoming", False),
**self.get_sport_specific_fields(game_event)
}
except Exception as e:
self.logger.error(f"Error extracting soccer game details: {e}")
return None
def get_sport_specific_fields(self, game_event: Dict) -> Dict:
"""Extract soccer-specific fields."""
try:
return {
"half": game_event.get("half", ""),
"stoppage_time": game_event.get("stoppage_time", ""),
"cards": {
"home_yellow": game_event.get("home_yellow_cards", 0),
"away_yellow": game_event.get("away_yellow_cards", 0),
"home_red": game_event.get("home_red_cards", 0),
"away_red": game_event.get("away_red_cards", 0)
},
"possession": {
"home": game_event.get("home_possession", 0),
"away": game_event.get("away_possession", 0)
}
}
except Exception as e:
self.logger.error(f"Error extracting soccer-specific fields: {e}")
return {}
# Factory function removed - sport classes now instantiate extractors directly
+692
View File
@@ -0,0 +1,692 @@
"""
Baseball Base Classes
This module provides baseball-specific base classes that extend the core sports functionality
with baseball-specific logic for innings, outs, bases, strikes, balls, etc.
"""
import logging
import time
from typing import Any, Dict, Optional
from PIL import Image, ImageDraw, ImageFont
from src.base_classes.data_sources import ESPNDataSource
from src.base_classes.sports import SportsCore, SportsLive, SportsRecent
class Baseball(SportsCore):
"""Base class for baseball sports with common functionality."""
def __init__(
self,
config: Dict[str, Any],
display_manager,
cache_manager,
logger: logging.Logger,
sport_key: str,
):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
# Baseball-specific configuration
self.show_innings = self.mode_config.get("show_innings", True)
self.show_outs = self.mode_config.get("show_outs", True)
self.show_bases = self.mode_config.get("show_bases", True)
self.show_count = self.mode_config.get("show_count", True)
self.show_pitcher_batter = self.mode_config.get("show_pitcher_batter", False)
self.show_series_summary = self.mode_config.get("show_series_summary", False)
self.data_source = ESPNDataSource(logger)
self.sport = "baseball"
def _is_baseball_game_live(self, game: Dict) -> bool:
"""Check if a baseball game is currently live."""
try:
# Check if game is marked as live
is_live = game.get("is_live", False)
if is_live:
return True
# Check inning to determine if game is active
inning = game.get("inning", "")
if inning and inning != "Final":
return True
return False
except Exception as e:
self.logger.error(f"Error checking if baseball game is live: {e}")
return False
def _get_baseball_game_status(self, game: Dict) -> str:
"""Get baseball-specific game status."""
try:
status = game.get("status_text", "")
inning = game.get("inning", "")
if self._is_baseball_game_live(game):
if inning:
return f"Live - {inning}"
else:
return "Live"
elif game.get("is_final", False):
return "Final"
elif game.get("is_upcoming", False):
return "Upcoming"
else:
return status
except Exception as e:
self.logger.error(f"Error getting baseball game status: {e}")
return ""
def _extract_game_details(self, game_event: Dict) -> Optional[Dict]:
"""Extract relevant game details from ESPN NCAA FB API response."""
details, home_team, away_team, status, situation = (
self._extract_game_details_common(game_event)
)
if details is None or home_team is None or away_team is None or status is None:
return
try:
# print(status["type"]["state"])
# exit()
game_status = status["type"]["name"].lower()
status_state = status["type"]["state"].lower()
# Get team abbreviations
home_abbr = home_team["team"]["abbreviation"]
away_abbr = away_team["team"]["abbreviation"]
# Check if this is a favorite team game
is_favorite_game = (
home_abbr in self.favorite_teams or away_abbr in self.favorite_teams
)
# Log all teams found for debugging
self.logger.debug(
f"Found game: {away_abbr} @ {home_abbr} (Status: {game_status}, State: {status_state})"
)
# Only log detailed information for favorite teams
if is_favorite_game:
# Use the validated competition-level `status` here too. MiLB
# events carry no event-level one, so this debug line raised a
# KeyError and dropped the very games it was meant to help
# diagnose -- and only for favourites, which is the worst way
# for it to fail.
self.logger.debug(f"Full status data: {status}")
self.logger.debug(f"Status type: {game_status}, State: {status_state}")
self.logger.debug(f"Status detail: {status['type'].get('detail', '')}")
self.logger.debug(
f"Status shortDetail: {status['type'].get('shortDetail', '')}"
)
series = game_event["competitions"][0].get("series", None)
series_summary = ""
if series:
series_summary = series.get("summary", "")
# Get game state information
if status_state == "in":
# For live games, get detailed state
# Use the competition-level `status` already validated by
# _extract_game_details_common. Real ESPN events duplicate
# status at the event top level, but MiLB events (synthesized
# from the MLB Stats API into an ESPN-like shape) populate
# only the competition-level one, so the top-level lookup
# raised a bare KeyError and dropped the event.
inning = status.get(
"period", 1
) # Get inning from status period
# Get inning information from status
status_detail = status["type"].get("detail", "").lower()
status_short = status["type"].get("shortDetail", "").lower()
if is_favorite_game:
self.logger.debug(
f"Raw status detail: {status['type'].get('detail')}"
)
self.logger.debug(
f"Raw status short: {status['type'].get('shortDetail')}"
)
# Determine inning half from status information
inning_half = "top" # Default
# Handle end of inning: next inning is top
if "end" in status_detail or "end" in status_short:
inning_half = "top"
inning = (
status.get("period", 1) + 1
) # Use period and increment for next inning
if is_favorite_game:
self.logger.debug(
f"Detected end of inning. Setting to Top {inning}"
)
# Handle middle of inning: next is bottom of current inning
elif "mid" in status_detail or "mid" in status_short:
inning_half = "bottom"
if is_favorite_game:
self.logger.debug(
f"Detected middle of inning. Setting to Bottom {inning}"
)
# Handle bottom of inning
elif (
"bottom" in status_detail
or "bot" in status_detail
or "bottom" in status_short
or "bot" in status_short
):
inning_half = "bottom"
if is_favorite_game:
self.logger.debug(f"Detected bottom of inning: {inning}")
# Handle top of inning
elif "top" in status_detail or "top" in status_short:
inning_half = "top"
if is_favorite_game:
self.logger.debug(f"Detected top of inning: {inning}")
if is_favorite_game:
self.logger.debug(f"Status detail: {status_detail}")
self.logger.debug(f"Status short: {status_short}")
self.logger.debug(f"Determined inning: {inning_half} {inning}")
# Get count and bases from situation
situation = game_event["competitions"][0].get("situation", {})
if is_favorite_game:
self.logger.debug(f"Full situation data: {situation}")
# Get count from the correct location in the API response
count = situation.get("count", {})
balls = count.get("balls", 0)
strikes = count.get("strikes", 0)
outs = situation.get("outs", 0)
# Add detailed logging for favorite team games
if is_favorite_game:
self.logger.debug(f"Full situation data: {situation}")
self.logger.debug(f"Count object: {count}")
self.logger.debug(
f"Raw count values - balls: {balls}, strikes: {strikes}"
)
self.logger.debug(f"Raw outs value: {outs}")
# Try alternative locations for count data
if balls == 0 and strikes == 0:
# First try the summary field
if "summary" in situation:
try:
count_summary = situation["summary"]
balls, strikes = map(int, count_summary.split("-"))
if is_favorite_game:
self.logger.debug(
f"Using summary count: {count_summary}"
)
except (ValueError, AttributeError):
if is_favorite_game:
self.logger.debug("Could not parse summary count")
else:
# Check if count is directly in situation
balls = situation.get("balls", 0)
strikes = situation.get("strikes", 0)
if is_favorite_game:
self.logger.debug(
f"Using direct situation count: balls={balls}, strikes={strikes}"
)
self.logger.debug(
f"Full situation keys: {list(situation.keys())}"
)
if is_favorite_game:
self.logger.debug(f"Final count: balls={balls}, strikes={strikes}")
# Get base runners
bases_occupied = [
situation.get("onFirst", False),
situation.get("onSecond", False),
situation.get("onThird", False),
]
if is_favorite_game:
self.logger.debug(f"Bases occupied: {bases_occupied}")
else:
# Default values for non-live games
inning = 1
inning_half = "top"
balls = 0
strikes = 0
outs = 0
bases_occupied = [False, False, False]
details.update(
{
"status": game_status,
"status_state": status_state,
"inning": inning,
"inning_half": inning_half,
"balls": balls,
"strikes": strikes,
"outs": outs,
"bases_occupied": bases_occupied,
"start_time": game_event["date"],
"series_summary": series_summary,
}
)
# Basic validation (can be expanded)
if not details["home_abbr"] or not details["away_abbr"]:
self.logger.warning(
f"Missing team abbreviation in event: {details['id']}"
)
return None
self.logger.debug(
f"Extracted: {details['away_abbr']}@{details['home_abbr']}, Status: {status['type']['name']}, Live: {details['is_live']}, Final: {details['is_final']}, Upcoming: {details['is_upcoming']}"
)
return details
except Exception as e:
# Log the problematic event structure if possible
self.logger.error(
f"Error extracting game details: {e} from event: {game_event.get('id')}",
exc_info=True,
)
return None
def display_series_summary(self, game: dict, draw_overlay: ImageDraw.ImageDraw):
if not self.show_series_summary:
return
series_summary = game.get("series_summary", "")
bbox = draw_overlay.textbbox((0, 0), series_summary, font=self.fonts['time'])
height = bbox[3] - bbox[1]
shots_y = (self.display_height - height) // 2
shots_width = draw_overlay.textlength(series_summary, font=self.fonts['time'])
shots_x = (self.display_width - shots_width) // 2
self._draw_text_with_outline(
draw_overlay, series_summary, (shots_x, shots_y), self.fonts['time']
)
class BaseballRecent(Baseball, SportsRecent):
"""Base class for recent baseball games."""
def __init__(
self,
config: Dict[str, Any],
display_manager,
cache_manager,
logger: logging.Logger,
sport_key: str,
):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
def _custom_scorebug_layout(self, game: dict, draw_overlay: ImageDraw.ImageDraw):
self.display_series_summary(game, draw_overlay)
class BaseballLive(Baseball, SportsLive):
"""Base class for live baseball games."""
def __init__(
self,
config: Dict[str, Any],
display_manager,
cache_manager,
logger: logging.Logger,
sport_key: str,
):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
def _test_mode_update(self):
if self.current_game and self.current_game["is_live"]:
# self.current_game["bases_occupied"] = [
# random.choice([True, False]) for _ in range(3)
# ]
# self.current_game["balls"] = random.choice([1, 2, 3])
# self.current_game["strikes"] = random.choice([1, 2])
# self.current_game["outs"] = random.choice([1, 2])
if self.current_game["inning_half"] == "top":
self.current_game["inning_half"] = "bottom"
else:
self.current_game["inning_half"] = "top"
self.current_game["inning"] += 1
self.current_game["balls"] = (self.current_game["balls"] + 1) % 4
self.current_game["strikes"] = (self.current_game["strikes"] + 1) % 3
self.current_game["outs"] = (self.current_game["outs"] + 1) % 3
self.current_game["bases_occupied"] = [
not b for b in self.current_game["bases_occupied"]
]
if self.current_game["inning"] % 2 == 0:
self.current_game["home_score"] = str(
int(self.current_game["home_score"]) + 1
)
else:
self.current_game["away_score"] = str(
int(self.current_game["away_score"]) + 1
)
def _draw_scorebug_layout(self, game: Dict, force_clear: bool = False) -> None:
"""Draw the detailed scorebug layout for a live NCAA FB game.""" # Updated docstring
try:
main_img = Image.new(
"RGBA", (self.display_width, self.display_height), (0, 0, 0, 255)
)
overlay = Image.new(
"RGBA", (self.display_width, self.display_height), (0, 0, 0, 0)
)
draw_overlay = ImageDraw.Draw(
overlay
) # Draw text elements on overlay first
home_logo = self._load_and_resize_logo(
game["home_id"],
game["home_abbr"],
game["home_logo_path"],
game.get("home_logo_url"),
)
away_logo = self._load_and_resize_logo(
game["away_id"],
game["away_abbr"],
game["away_logo_path"],
game.get("away_logo_url"),
)
if not home_logo or not away_logo:
self.logger.error(
f"Failed to load logos for live game: {game.get('id')}"
) # Changed log prefix
# Draw placeholder text if logos fail
draw_final = ImageDraw.Draw(main_img.convert("RGB"))
self._draw_text_with_outline(
draw_final, "Logo Error", (5, 5), self.fonts["status"]
)
self.display_manager.image.paste(main_img.convert("RGB"), (0, 0))
self.display_manager.update_display()
return
center_y = self.display_height // 2
# Draw logos (shifted slightly more inward than NHL perhaps)
home_x = (
self.display_width - home_logo.width + 10
) # adjusted from 18 # Adjust position as needed
home_y = center_y - (home_logo.height // 2)
main_img.paste(home_logo, (home_x, home_y), home_logo)
away_x = -10 # adjusted from 18 # Adjust position as needed
away_y = center_y - (away_logo.height // 2)
main_img.paste(away_logo, (away_x, away_y), away_logo)
# --- Live Game Specific Elements ---
# Define default text color
text_color = (255, 255, 255)
# Draw Inning (Top Center)
inning_half = game["inning_half"]
inning_num = game["inning"]
if game["is_final"]:
inning_text = "FINAL"
else:
inning_half_indicator = (
"▲" if game["inning_half"].lower() == "top" else "▼"
)
inning_num = game["inning"]
inning_text = f"{inning_half_indicator}{inning_num}"
inning_bbox = draw_overlay.textbbox(
(0, 0), inning_text, font=self.display_manager.font
)
inning_width = inning_bbox[2] - inning_bbox[0]
inning_x = (self.display_width - inning_width) // 2
inning_y = 1 # Position near top center
# draw_overlay.text((inning_x, inning_y), inning_text, fill=(255, 255, 255), font=self.display_manager.font)
self._draw_text_with_outline(
draw_overlay,
inning_text,
(inning_x, inning_y),
self.display_manager.font,
)
# --- REVISED BASES AND OUTS DRAWING ---
bases_occupied = game["bases_occupied"] # [1st, 2nd, 3rd]
outs = game.get("outs", 0)
inning_half = game["inning_half"]
# Define geometry
base_diamond_size = 7
out_circle_diameter = 3
out_vertical_spacing = 2 # Space between out circles
spacing_between_bases_outs = (
3 # Horizontal space between base cluster and out column
)
base_vert_spacing = 1 # Internal vertical space in base cluster
base_horiz_spacing = 1 # Internal horizontal space in base cluster
# Calculate cluster dimensions
base_cluster_height = (
base_diamond_size + base_vert_spacing + base_diamond_size
)
base_cluster_width = (
base_diamond_size + base_horiz_spacing + base_diamond_size
)
out_cluster_height = 3 * out_circle_diameter + 2 * out_vertical_spacing
out_cluster_width = out_circle_diameter
# Calculate overall start positions
overall_start_y = (
inning_bbox[3] + 0
) # Start immediately below inning text (moved up 3 pixels)
# Center the BASE cluster horizontally
bases_origin_x = (self.display_width - base_cluster_width) // 2
# Determine relative positions for outs based on inning half
if inning_half == "top": # Away batting, outs on left
outs_column_x = (
bases_origin_x - spacing_between_bases_outs - out_cluster_width
)
else: # Home batting, outs on right
outs_column_x = (
bases_origin_x + base_cluster_width + spacing_between_bases_outs
)
# Calculate vertical alignment offset for outs column (center align with bases cluster)
outs_column_start_y = (
overall_start_y + (base_cluster_height // 2) - (out_cluster_height // 2)
)
# --- Draw Bases (Diamonds) ---
base_color_occupied = (255, 255, 255)
base_color_empty = (255, 255, 255) # Outline color
h_d = base_diamond_size // 2
# 2nd Base (Top center relative to bases_origin_x)
c2x = bases_origin_x + base_cluster_width // 2
c2y = overall_start_y + h_d
poly2 = [
(c2x, overall_start_y),
(c2x + h_d, c2y),
(c2x, c2y + h_d),
(c2x - h_d, c2y),
]
if bases_occupied[1]:
draw_overlay.polygon(poly2, fill=base_color_occupied)
else:
draw_overlay.polygon(poly2, outline=base_color_empty)
base_bottom_y = c2y + h_d # Bottom Y of 2nd base diamond
# 3rd Base (Bottom left relative to bases_origin_x)
c3x = bases_origin_x + h_d
c3y = base_bottom_y + base_vert_spacing + h_d
poly3 = [
(c3x, base_bottom_y + base_vert_spacing),
(c3x + h_d, c3y),
(c3x, c3y + h_d),
(c3x - h_d, c3y),
]
if bases_occupied[2]:
draw_overlay.polygon(poly3, fill=base_color_occupied)
else:
draw_overlay.polygon(poly3, outline=base_color_empty)
# 1st Base (Bottom right relative to bases_origin_x)
c1x = bases_origin_x + base_cluster_width - h_d
c1y = base_bottom_y + base_vert_spacing + h_d
poly1 = [
(c1x, base_bottom_y + base_vert_spacing),
(c1x + h_d, c1y),
(c1x, c1y + h_d),
(c1x - h_d, c1y),
]
if bases_occupied[0]:
draw_overlay.polygon(poly1, fill=base_color_occupied)
else:
draw_overlay.polygon(poly1, outline=base_color_empty)
# --- Draw Outs (Vertical Circles) ---
circle_color_out = (255, 255, 255)
circle_color_empty_outline = (100, 100, 100)
for i in range(3):
cx = outs_column_x
cy = outs_column_start_y + i * (
out_circle_diameter + out_vertical_spacing
)
coords = [cx, cy, cx + out_circle_diameter, cy + out_circle_diameter]
if i < outs:
draw_overlay.ellipse(coords, fill=circle_color_out)
else:
draw_overlay.ellipse(coords, outline=circle_color_empty_outline)
# --- Draw Balls-Strikes Count (BDF Font) ---
balls = game.get("balls", 0)
strikes = game.get("strikes", 0)
# Add debug logging for count with cooldown
current_time = time.time()
if (
game["home_abbr"] in self.favorite_teams
or game["away_abbr"] in self.favorite_teams
) and current_time - self.last_count_log_time >= self.count_log_interval:
self.logger.debug(f"Displaying count: {balls}-{strikes}")
self.logger.debug(
f"Raw count data: balls={game.get('balls')}, strikes={game.get('strikes')}"
)
self.last_count_log_time = current_time
count_text = f"{balls}-{strikes}"
bdf_font = self.display_manager.calendar_font
bdf_font.set_char_size(height=7 * 64) # Set 7px height
count_text_width = self.display_manager.get_text_width(count_text, bdf_font)
# Position below the base/out cluster
cluster_bottom_y = (
overall_start_y + base_cluster_height
) # Find the bottom of the taller part (bases)
count_y = cluster_bottom_y + 2 # Start 2 pixels below cluster
# Center horizontally within the BASE cluster width
count_x = bases_origin_x + (base_cluster_width - count_text_width) // 2
# Ensure draw object is set and draw text
self.display_manager.draw = draw_overlay
# self.display_manager._draw_bdf_text(count_text, count_x, count_y, text_color, font=bdf_font)
# Use _draw_text_with_outline for count text
# self._draw_text_with_outline(draw, count_text, (count_x, count_y), bdf_font, fill=text_color)
# Draw Balls-Strikes Count with outline using BDF font
# Define outline color (consistent with _draw_text_with_outline default)
outline_color_for_bdf = (0, 0, 0)
# Draw outline
for dx_offset, dy_offset in [
(-1, -1),
(-1, 0),
(-1, 1),
(0, -1),
(0, 1),
(1, -1),
(1, 0),
(1, 1),
]:
self.display_manager._draw_bdf_text(
count_text,
count_x + dx_offset,
count_y + dy_offset,
color=outline_color_for_bdf,
font=bdf_font,
)
# Draw main text
self.display_manager._draw_bdf_text(
count_text, count_x, count_y, color=text_color, font=bdf_font
)
# Draw Team:Score at the bottom (matching main branch format)
score_font = self.display_manager.font # Use PressStart2P
outline_color = (0, 0, 0)
score_text_color = (
255,
255,
255,
) # Use a specific name for score text color
# Helper function for outlined text
def draw_bottom_outlined_text(x, y, text):
self._draw_text_with_outline(
draw_overlay,
text,
(x, y),
score_font,
fill=score_text_color,
outline_color=outline_color,
)
away_abbr = game["away_abbr"]
home_abbr = game["home_abbr"]
away_score_str = str(game["away_score"])
home_score_str = str(game["home_score"])
away_text = f"{away_abbr}:{away_score_str}"
home_text = f"{home_abbr}:{home_score_str}"
# Calculate Y position (bottom edge)
# Get font height (approximate or precise)
try:
font_height = score_font.getbbox("A")[3] - score_font.getbbox("A")[1]
except AttributeError:
font_height = 8 # Fallback for default font
score_y = (
self.display_height - font_height - 2
) # 2 pixels padding from bottom
# Away Team:Score (Bottom Left)
away_score_x = 2 # 2 pixels padding from left
draw_bottom_outlined_text(away_score_x, score_y, away_text)
# Home Team:Score (Bottom Right)
home_text_bbox = draw_overlay.textbbox((0, 0), home_text, font=score_font)
home_text_width = home_text_bbox[2] - home_text_bbox[0]
home_score_x = (
self.display_width - home_text_width - 2
) # 2 pixels padding from right
draw_bottom_outlined_text(home_score_x, score_y, home_text)
# Draw gambling odds if available
if "odds" in game and game["odds"]:
self._draw_dynamic_odds(
draw_overlay, game["odds"], self.display_width, self.display_height
)
# Composite the text overlay onto the main image
main_img = Image.alpha_composite(main_img, overlay)
main_img = main_img.convert("RGB") # Convert for display
# Display the final image
self.display_manager.image.paste(main_img, (0, 0))
self.display_manager.update_display() # Update display here for live
except Exception as e:
self.logger.error(
f"Error displaying live Baseball game: {e}", exc_info=True
) # Changed log prefix
+300
View File
@@ -0,0 +1,300 @@
import logging
from typing import Any, Dict, Optional
from PIL import Image, ImageDraw, ImageFont
from src.base_classes.data_sources import ESPNDataSource
from src.base_classes.sports import SportsCore, SportsLive
from src.cache_manager import CacheManager
from src.display_manager import DisplayManager
class Basketball(SportsCore):
"""Base class for basketball sports with common functionality."""
def __init__(
self,
config: Dict[str, Any],
display_manager: DisplayManager,
cache_manager: CacheManager,
logger: logging.Logger,
sport_key: str,
):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
self.data_source = ESPNDataSource(logger)
self.sport = "basketball"
def _extract_game_details(self, game_event: Dict) -> Optional[Dict]:
"""Extract relevant game details from ESPN NCAA FB API response."""
# --- THIS METHOD MAY NEED ADJUSTMENTS FOR NCAA FB API DIFFERENCES ---
details, home_team, away_team, status, situation = (
self._extract_game_details_common(game_event)
)
if details is None or home_team is None or away_team is None or status is None:
return
try:
# Format period/quarter
period = status.get("period", 0)
period_text = ""
if status["type"]["state"] == "in":
if period == 0:
period_text = "Start" # Before kickoff
elif period >= 1 and period <= 4:
period_text = f"Q{period}" # OT starts after Q4
elif period > 4:
period_text = f"OT{period - 4}" # OT starts after Q4
elif status["type"]["state"] == "halftime" or status["type"]["name"] == "STATUS_HALFTIME": # Check explicit halftime state
period_text = "HALF"
elif status["type"]["state"] == "post":
if period > 4 : period_text = "Final/OT"
else: period_text = "Final"
elif status["type"]["state"] == "pre":
period_text = details.get("game_time", "") # Show time for upcoming
details.update({
"period": period,
"period_text": period_text, # Formatted quarter/status
"clock": status.get("displayClock", "0:00"),
})
# Basic validation (can be expanded)
if not details["home_abbr"] or not details["away_abbr"]:
self.logger.warning(
f"Missing team abbreviation in event: {details['id']}"
)
return None
self.logger.debug(
f"Extracted: {details['away_abbr']}@{details['home_abbr']}, Status: {status['type']['name']}, Live: {details['is_live']}, Final: {details['is_final']}, Upcoming: {details['is_upcoming']}"
)
return details
except Exception as e:
# Log the problematic event structure if possible
self.logger.error(
f"Error extracting game details: {e} from event: {game_event.get('id')}",
exc_info=True,
)
return None
class BasketballLive(Basketball, SportsLive):
def __init__(
self,
config: Dict[str, Any],
display_manager: DisplayManager,
cache_manager: CacheManager,
logger: logging.Logger,
sport_key: str,
):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
def _test_mode_update(self):
if self.current_game and self.current_game["is_live"]:
# For testing, we'll just update the clock to show it's working
minutes = int(self.current_game["clock"].split(":")[0])
seconds = int(self.current_game["clock"].split(":")[1])
seconds -= 1
if seconds < 0:
seconds = 59
minutes -= 1
if minutes < 0:
minutes = 19
if self.current_game["period"] < 3:
self.current_game["period"] += 1
else:
self.current_game["period"] = 1
self.current_game["clock"] = f"{minutes:02d}:{seconds:02d}"
# Always update display in test mode
def _draw_scorebug_layout(self, game: Dict, force_clear: bool = False) -> None:
"""Draw the detailed scorebug layout for a live Basketball game.""" # Updated docstring
try:
main_img = Image.new(
"RGBA", (self.display_width, self.display_height), (0, 0, 0, 255)
)
overlay = Image.new(
"RGBA", (self.display_width, self.display_height), (0, 0, 0, 0)
)
draw_overlay = ImageDraw.Draw(
overlay
) # Draw text elements on overlay first
home_logo = self._load_and_resize_logo(
game["home_id"],
game["home_abbr"],
game["home_logo_path"],
game.get("home_logo_url"),
)
away_logo = self._load_and_resize_logo(
game["away_id"],
game["away_abbr"],
game["away_logo_path"],
game.get("away_logo_url"),
)
if not home_logo or not away_logo:
self.logger.error(
f"Failed to load logos for live game: {game.get('id')}"
) # Changed log prefix
# Draw placeholder text if logos fail
draw_final = ImageDraw.Draw(main_img.convert("RGB"))
self._draw_text_with_outline(
draw_final, "Logo Error", (5, 5), self.fonts["status"]
)
self.display_manager.image.paste(main_img.convert("RGB"), (0, 0))
self.display_manager.update_display()
return
center_y = self.display_height // 2
# Draw logos (shifted slightly more inward than NHL perhaps)
home_x = (
self.display_width - home_logo.width + 10
) # adjusted from 18 # Adjust position as needed
home_y = center_y - (home_logo.height // 2)
main_img.paste(home_logo, (home_x, home_y), home_logo)
away_x = -10 # adjusted from 18 # Adjust position as needed
away_y = center_y - (away_logo.height // 2)
main_img.paste(away_logo, (away_x, away_y), away_logo)
# --- Draw Text Elements on Overlay ---
# Note: Rankings are now handled in the records/rankings section below
# Period/Quarter and Clock (Top center)
period_clock_text = (
f"{game.get('period_text', '')} {game.get('clock', '')}".strip()
)
status_width = draw_overlay.textlength(
period_clock_text, font=self.fonts["time"]
)
status_x = (self.display_width - status_width) // 2
status_y = 1 # Position at top
self._draw_text_with_outline(
draw_overlay,
period_clock_text,
(status_x, status_y),
self.fonts["time"],
)
# Scores (centered, slightly above bottom)
home_score = str(game.get("home_score", "0"))
away_score = str(game.get("away_score", "0"))
score_text = f"{away_score}-{home_score}"
score_width = draw_overlay.textlength(score_text, font=self.fonts["score"])
score_x = (self.display_width - score_width) // 2
score_y = (
self.display_height // 2
) - 3 # centered #from 14 # Position score higher
self._draw_text_with_outline(
draw_overlay, score_text, (score_x, score_y), self.fonts["score"]
)
# Draw odds if available
if "odds" in game and game["odds"]:
self._draw_dynamic_odds(
draw_overlay, game["odds"], self.display_width, self.display_height
)
# Draw records or rankings if enabled
if self.show_records or self.show_ranking:
record_font = self.fonts.get('detail', ImageFont.load_default())
# Get team abbreviations
away_abbr = game.get("away_abbr", "")
home_abbr = game.get("home_abbr", "")
record_bbox = draw_overlay.textbbox((0, 0), "0-0", font=record_font)
record_height = record_bbox[3] - record_bbox[1]
record_y = self.display_height - record_height - 1
self.logger.debug(
f"Record positioning: height={record_height}, record_y={record_y}, display_height={self.display_height}"
)
# Display away team info
if away_abbr:
if self.show_ranking and self.show_records:
# When both rankings and records are enabled, rankings replace records completely
away_rank = self._team_rankings_cache.get(away_abbr, 0)
if away_rank > 0:
away_text = f"#{away_rank}"
else:
# Show nothing for unranked teams when rankings are prioritized
away_text = ""
elif self.show_ranking:
# Show ranking only if available
away_rank = self._team_rankings_cache.get(away_abbr, 0)
if away_rank > 0:
away_text = f"#{away_rank}"
else:
away_text = ""
elif self.show_records:
# Show record only when rankings are disabled
away_text = game.get("away_record", "")
else:
away_text = ""
if away_text:
away_record_x = 3
self.logger.debug(
f"Drawing away ranking '{away_text}' at ({away_record_x}, {record_y}) with font size {record_font.size if hasattr(record_font, 'size') else 'unknown'}"
)
self._draw_text_with_outline(
draw_overlay,
away_text,
(away_record_x, record_y),
record_font,
)
# Display home team info
if home_abbr:
if self.show_ranking and self.show_records:
# When both rankings and records are enabled, rankings replace records completely
home_rank = self._team_rankings_cache.get(home_abbr, 0)
if home_rank > 0:
home_text = f"#{home_rank}"
else:
# Show nothing for unranked teams when rankings are prioritized
home_text = ""
elif self.show_ranking:
# Show ranking only if available
home_rank = self._team_rankings_cache.get(home_abbr, 0)
if home_rank > 0:
home_text = f"#{home_rank}"
else:
home_text = ""
elif self.show_records:
# Show record only when rankings are disabled
home_text = game.get("home_record", "")
else:
home_text = ""
if home_text:
home_record_bbox = draw_overlay.textbbox(
(0, 0), home_text, font=record_font
)
home_record_width = home_record_bbox[2] - home_record_bbox[0]
home_record_x = self.display_width - home_record_width - 3
self.logger.debug(
f"Drawing home ranking '{home_text}' at ({home_record_x}, {record_y}) with font size {record_font.size if hasattr(record_font, 'size') else 'unknown'}"
)
self._draw_text_with_outline(
draw_overlay,
home_text,
(home_record_x, record_y),
record_font,
)
# Composite the text overlay onto the main image
main_img = Image.alpha_composite(main_img, overlay)
main_img = main_img.convert("RGB") # Convert for display
# Display the final image
self.display_manager.image.paste(main_img, (0, 0))
self.display_manager.update_display() # Update display here for live
except Exception as e:
self.logger.error(
f"Error displaying live Hockey game: {e}", exc_info=True
) # Changed log prefix
+344
View File
@@ -0,0 +1,344 @@
"""
Pluggable Data Source Architecture
This module provides abstract data sources that can be plugged into the sports system
to support different APIs and data providers.
"""
from abc import ABC, abstractmethod
from typing import Dict, List
import requests
import logging
from datetime import datetime
from src.common.espn_dates import fetch_espn_scoreboard
class DataSource(ABC):
"""Abstract base class for data sources."""
def __init__(self, logger: logging.Logger):
self.logger = logger
self.session = requests.Session()
# Configure retry strategy
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
retry_strategy = Retry(
total=5,
backoff_factor=1,
status_forcelist=[429, 500, 502, 503, 504],
)
adapter = HTTPAdapter(max_retries=retry_strategy)
self.session.mount("http://", adapter)
self.session.mount("https://", adapter)
@abstractmethod
def fetch_live_games(self, sport: str, league: str) -> List[Dict]:
"""Fetch live games for a sport/league."""
@abstractmethod
def fetch_schedule(self, sport: str, league: str, date_range: tuple) -> List[Dict]:
"""Fetch schedule for a sport/league within date range."""
@abstractmethod
def fetch_standings(self, sport: str, league: str) -> Dict:
"""Fetch standings for a sport/league."""
def get_headers(self) -> Dict[str, str]:
"""Get headers for API requests.
The agent carries the project URL deliberately. Around 2026-08-04 ESPN
began returning 403 for bare custom tokens like 'LEDMatrix/1.0' — and
for browser-style strings — while accepting an agent that identifies
the client and links to it. An Accept header alone does not rescue the
bare form when the request goes out through requests.
"""
return {
'User-Agent': 'LEDMatrix/1.0 (+https://github.com/ChuckBuilds/LEDMatrix)',
'Accept': 'application/json'
}
class ESPNDataSource(DataSource):
"""ESPN API data source."""
def __init__(self, logger: logging.Logger):
super().__init__(logger)
self.base_url = "https://site.api.espn.com/apis/site/v2/sports"
def fetch_live_games(self, sport: str, league: str) -> List[Dict]:
"""Fetch live games from ESPN API."""
try:
now = datetime.now()
formatted_date = now.strftime("%Y%m%d")
url = f"{self.base_url}/{sport}/{league}/scoreboard"
data = fetch_espn_scoreboard(
self.session, url, params={"dates": formatted_date, "limit": 1000},
headers=self.get_headers(), timeout=15, logger=self.logger,
)
events = data.get('events', [])
# Filter for live games
live_events = [event for event in events
if event.get('competitions', [{}])[0].get('status', {}).get('type', {}).get('state') == 'in']
self.logger.debug(f"Fetched {len(live_events)} live games for {sport}/{league}")
return live_events
except Exception as e:
self.logger.error(f"Error fetching live games from ESPN: {e}")
return []
def fetch_schedule(self, sport: str, league: str, date_range: tuple) -> List[Dict]:
"""Fetch schedule from ESPN API."""
try:
start_date, end_date = date_range
url = f"{self.base_url}/{sport}/{league}/scoreboard"
params = {
'dates': f"{start_date.strftime('%Y%m%d')}-{end_date.strftime('%Y%m%d')}",
"limit": 1000
}
data = fetch_espn_scoreboard(
self.session, url, params=params,
headers=self.get_headers(), timeout=15, logger=self.logger,
)
events = data.get('events', [])
self.logger.debug(f"Fetched {len(events)} scheduled games for {sport}/{league}")
return events
except Exception as e:
self.logger.error(f"Error fetching schedule from ESPN: {e}")
return []
def fetch_standings(self, sport: str, league: str) -> Dict:
"""Fetch standings, or the poll for leagues that have one.
Order matters and used to be wrong. College leagues publish a poll at
/rankings and a records table at /standings; professional leagues have
only /standings. The old code tried /standings first and fell back to
/rankings only on a 404 -- but college /standings answers 200, so the
fallback never fired and college rankings came back empty forever.
Nothing failed; the AP rank badge simply never appeared, and anything
else keyed off rankings quietly did nothing.
A 200 that lacks the key is treated as a miss, so a league answering
both endpoints still ends up with whichever one actually carries a poll.
"""
league_name = (league or "").lower()
wants_poll = "college" in league_name or "ncaa" in league_name
endpoints = ["rankings", "standings"] if wants_poll else ["standings", "rankings"]
for endpoint in endpoints:
url = f"{self.base_url}/{sport}/{league}/{endpoint}"
# Only the request is guarded. Inspecting the payload happens
# below, outside the handler, so that a bug in this method cannot
# be mistaken for an endpoint that failed -- that mistake would
# silently drop rankings for a league that has them, which is the
# exact failure this function was written to fix.
try:
response = self.session.get(
url, headers=self.get_headers(), timeout=15
)
response.raise_for_status()
data = response.json()
except (requests.RequestException, ValueError) as e:
status = getattr(getattr(e, "response", None), "status_code", None)
# Only a 404 is routine -- it is how a league says "no poll
# here". Everything else is worth an error, and `status is
# None` covers the ones that matter most: ConnectionError,
# Timeout, a body that would not parse. Silencing those left a
# board that could not reach ESPN with one debug line, and the
# ranked filter running on an empty table.
if status != 404:
self.logger.error(
f"Error fetching {endpoint} from ESPN for "
f"{sport}/{league}: {e}"
)
continue
if not isinstance(data, dict):
# A list or a bare string is not something the callers can
# read. Treat it as a miss so the other endpoint still gets a
# turn, but say so -- this means ESPN changed shape.
self.logger.error(
f"Unexpected {endpoint} payload for {sport}/{league}: "
f"got {type(data).__name__}, expected an object"
)
continue
if endpoint == "rankings" and not data.get("rankings"):
continue
self.logger.debug(f"Fetched {endpoint} for {sport}/{league}")
return data
self.logger.debug(
f"Standings/rankings not available for {sport}/{league} from ESPN API"
)
return {}
class MLBAPIDataSource(DataSource):
"""MLB API data source."""
def __init__(self, logger: logging.Logger):
super().__init__(logger)
self.base_url = "https://statsapi.mlb.com/api/v1"
def fetch_live_games(self, sport: str, league: str) -> List[Dict]:
"""Fetch live games from MLB API."""
try:
url = f"{self.base_url}/schedule"
params = {
'sportId': 1, # MLB
'date': datetime.now().strftime('%Y-%m-%d'),
'hydrate': 'game,team,venue,weather'
}
response = self.session.get(url, headers=self.get_headers(), params=params, timeout=15)
response.raise_for_status()
data = response.json()
games = data.get('dates', [{}])[0].get('games', [])
# Filter for live games
live_games = [game for game in games
if game.get('status', {}).get('abstractGameState') == 'Live']
self.logger.debug(f"Fetched {len(live_games)} live games from MLB API")
return live_games
except Exception as e:
self.logger.error(f"Error fetching live games from MLB API: {e}")
return []
def fetch_schedule(self, sport: str, league: str, date_range: tuple) -> List[Dict]:
"""Fetch schedule from MLB API."""
try:
start_date, end_date = date_range
url = f"{self.base_url}/schedule"
params = {
'sportId': 1, # MLB
'startDate': start_date.strftime('%Y-%m-%d'),
'endDate': end_date.strftime('%Y-%m-%d'),
'hydrate': 'game,team,venue'
}
response = self.session.get(url, headers=self.get_headers(), params=params, timeout=15)
response.raise_for_status()
data = response.json()
all_games = []
for date_data in data.get('dates', []):
all_games.extend(date_data.get('games', []))
self.logger.debug(f"Fetched {len(all_games)} scheduled games from MLB API")
return all_games
except Exception as e:
self.logger.error(f"Error fetching schedule from MLB API: {e}")
return []
def fetch_standings(self, sport: str, league: str) -> Dict:
"""Fetch standings from MLB API."""
try:
url = f"{self.base_url}/standings"
params = {
'leagueId': 103, # American League
'season': datetime.now().year,
'standingsType': 'regularSeason'
}
response = self.session.get(url, headers=self.get_headers(), params=params, timeout=15)
response.raise_for_status()
data = response.json()
self.logger.debug("Fetched standings from MLB API")
return data
except Exception as e:
self.logger.error(f"Error fetching standings from MLB API: {e}")
return {}
class SoccerAPIDataSource(DataSource):
"""Soccer API data source (generic structure)."""
def __init__(self, logger: logging.Logger, api_key: str = None):
super().__init__(logger)
self.api_key = api_key
self.base_url = "https://api.football-data.org/v4" # Example API
def get_headers(self) -> Dict[str, str]:
"""Get headers with API key for soccer API."""
headers = super().get_headers()
if self.api_key:
headers['X-Auth-Token'] = self.api_key
return headers
def fetch_live_games(self, sport: str, league: str) -> List[Dict]:
"""Fetch live games from soccer API."""
try:
# This would need to be adapted based on the specific soccer API
url = f"{self.base_url}/matches"
params = {
'status': 'LIVE',
'competition': league
}
response = self.session.get(url, headers=self.get_headers(), params=params, timeout=15)
response.raise_for_status()
data = response.json()
matches = data.get('matches', [])
self.logger.debug(f"Fetched {len(matches)} live games from soccer API")
return matches
except Exception as e:
self.logger.error(f"Error fetching live games from soccer API: {e}")
return []
def fetch_schedule(self, sport: str, league: str, date_range: tuple) -> List[Dict]:
"""Fetch schedule from soccer API."""
try:
start_date, end_date = date_range
url = f"{self.base_url}/matches"
params = {
'competition': league,
'dateFrom': start_date.strftime('%Y-%m-%d'),
'dateTo': end_date.strftime('%Y-%m-%d')
}
response = self.session.get(url, headers=self.get_headers(), params=params, timeout=15)
response.raise_for_status()
data = response.json()
matches = data.get('matches', [])
self.logger.debug(f"Fetched {len(matches)} scheduled games from soccer API")
return matches
except Exception as e:
self.logger.error(f"Error fetching schedule from soccer API: {e}")
return []
def fetch_standings(self, sport: str, league: str) -> Dict:
"""Fetch standings from soccer API."""
try:
url = f"{self.base_url}/competitions/{league}/standings"
response = self.session.get(url, headers=self.get_headers(), timeout=15)
response.raise_for_status()
data = response.json()
self.logger.debug("Fetched standings from soccer API")
return data
except Exception as e:
self.logger.error(f"Error fetching standings from soccer API: {e}")
return {}
# Factory function removed - sport classes now instantiate data sources directly
+387
View File
@@ -0,0 +1,387 @@
from typing import Dict, Any, Optional
from src.display_manager import DisplayManager
from src.cache_manager import CacheManager
import logging
from PIL import Image, ImageDraw, ImageFont
from src.base_classes.data_sources import ESPNDataSource
from src.base_classes.sports import SportsCore, SportsLive
class Football(SportsCore):
"""Base class for football sports with common functionality."""
def __init__(self, config: Dict[str, Any], display_manager: DisplayManager, cache_manager: CacheManager, logger: logging.Logger, sport_key: str):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
self.data_source = ESPNDataSource(logger)
self.sport = "football"
def _extract_game_details(self, game_event: Dict) -> Optional[Dict]:
"""Extract relevant game details from ESPN NCAA FB API response."""
details, home_team, away_team, status, situation = self._extract_game_details_common(game_event)
if details is None or home_team is None or away_team is None or status is None:
return
try:
competition = game_event["competitions"][0]
status = competition["status"]
# --- Football Specific Details (Likely same for NFL/NCAAFB) ---
down_distance_text = ""
down_distance_text_long = ""
possession_indicator = None # Default to None
scoring_event = "" # Track scoring events
home_timeouts = 0
away_timeouts = 0
is_redzone = False
posession = None
if situation and status["type"]["state"] == "in":
# down = situation.get("down")
down_distance_text = situation.get("shortDownDistanceText")
down_distance_text_long = situation.get("downDistanceText")
# distance = situation.get("distance")
# Detect scoring events from status detail
status_detail = status["type"].get("detail", "").lower()
status_short = status["type"].get("shortDetail", "").lower()
is_redzone = situation.get("isRedZone")
posession = situation.get("possession")
# Check for scoring events in status text
if any(keyword in status_detail for keyword in ["touchdown", "td"]):
scoring_event = "TOUCHDOWN"
elif any(keyword in status_detail for keyword in ["field goal", "fg"]):
scoring_event = "FIELD GOAL"
elif any(keyword in status_detail for keyword in ["extra point", "pat", "point after"]):
scoring_event = "PAT"
elif any(keyword in status_short for keyword in ["touchdown", "td"]):
scoring_event = "TOUCHDOWN"
elif any(keyword in status_short for keyword in ["field goal", "fg"]):
scoring_event = "FIELD GOAL"
elif any(keyword in status_short for keyword in ["extra point", "pat"]):
scoring_event = "PAT"
# Determine possession based on team ID
possession_team_id = situation.get("possession")
if possession_team_id:
if possession_team_id == home_team.get("id"):
possession_indicator = "home"
elif possession_team_id == away_team.get("id"):
possession_indicator = "away"
home_timeouts = situation.get("homeTimeouts", 3) # Default to 3 if not specified
away_timeouts = situation.get("awayTimeouts", 3) # Default to 3 if not specified
# Format period/quarter
period = status.get("period", 0)
period_text = ""
if status["type"]["state"] == "in":
if period == 0:
period_text = "Start" # Before kickoff
elif period >= 1 and period <= 4:
period_text = f"Q{period}" # OT starts after Q4
elif period > 4:
period_text = f"OT{period - 4}" # OT starts after Q4
elif status["type"]["state"] == "halftime" or status["type"]["name"] == "STATUS_HALFTIME": # Check explicit halftime state
period_text = "HALF"
elif status["type"]["state"] == "post":
if period > 4 : period_text = "Final/OT"
else: period_text = "Final"
elif status["type"]["state"] == "pre":
period_text = details.get("game_time", "") # Show time for upcoming
details.update({
"period": period,
"period_text": period_text, # Formatted quarter/status
"clock": status.get("displayClock", "0:00"),
"home_timeouts": home_timeouts,
"away_timeouts": away_timeouts,
"down_distance_text": down_distance_text, # Added Down/Distance
"down_distance_text_long": down_distance_text_long,
"is_redzone": is_redzone,
"possession": posession, # ID of team with possession
"possession_indicator": possession_indicator, # Added for easy home/away check
"scoring_event": scoring_event, # Track scoring events (TOUCHDOWN, FIELD GOAL, PAT)
})
# Basic validation (can be expanded)
if not details['home_abbr'] or not details['away_abbr']:
self.logger.warning(f"Missing team abbreviation in event: {details['id']}")
return None
self.logger.debug(f"Extracted: {details['away_abbr']}@{details['home_abbr']}, Status: {status['type']['name']}, Live: {details['is_live']}, Final: {details['is_final']}, Upcoming: {details['is_upcoming']}")
return details
except Exception as e:
# Log the problematic event structure if possible
logging.error(f"Error extracting game details: {e} from event: {game_event.get('id')}", exc_info=True)
return None
class FootballLive(Football, SportsLive):
def __init__(self, config: Dict[str, Any], display_manager: DisplayManager, cache_manager: CacheManager, logger: logging.Logger, sport_key: str):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
def _test_mode_update(self):
if self.current_game and self.current_game["is_live"]:
try:
minutes, seconds = map(int, self.current_game["clock"].split(':'))
seconds -= 1
if seconds < 0:
seconds = 59
minutes -= 1
if minutes < 0:
# Simulate end of quarter/game
if self.current_game["period"] < 4: # Q4 is period 4
self.current_game["period"] += 1
# Update period_text based on new period
if self.current_game["period"] == 1: self.current_game["period_text"] = "Q1"
elif self.current_game["period"] == 2: self.current_game["period_text"] = "Q2"
elif self.current_game["period"] == 3: self.current_game["period_text"] = "Q3"
elif self.current_game["period"] == 4: self.current_game["period_text"] = "Q4"
# Reset clock for next quarter (e.g., 15:00)
minutes, seconds = 15, 0
else:
# Simulate game end
self.current_game["is_live"] = False
self.current_game["is_final"] = True
self.current_game["period_text"] = "Final"
minutes, seconds = 0, 0
self.current_game["clock"] = f"{minutes:02d}:{seconds:02d}"
# Simulate down change occasionally
if seconds % 15 == 0:
self.current_game["down_distance_text"] = f"{['1st','2nd','3rd','4th'][seconds % 4]} & {seconds % 10 + 1}"
self.current_game["status_text"] = f"{self.current_game['period_text']} {self.current_game['clock']}"
# Display update handled by main loop or explicit call if needed immediately
# self.display(force_clear=True) # Only if immediate update is desired here
except ValueError:
self.logger.warning("Test mode: Could not parse clock") # Changed log prefix
# No actual display call here, let main loop handle it
def _draw_scorebug_layout(self, game: Dict, force_clear: bool = False) -> None:
"""Draw the detailed scorebug layout for a live NCAA FB game.""" # Updated docstring
try:
main_img = Image.new('RGBA', (self.display_width, self.display_height), (0, 0, 0, 255))
overlay = Image.new('RGBA', (self.display_width, self.display_height), (0, 0, 0, 0))
draw_overlay = ImageDraw.Draw(overlay) # Draw text elements on overlay first
home_logo = self._load_and_resize_logo(game["home_id"], game["home_abbr"], game["home_logo_path"], game.get("home_logo_url"))
away_logo = self._load_and_resize_logo(game["away_id"], game["away_abbr"], game["away_logo_path"], game.get("away_logo_url"))
if not home_logo or not away_logo:
self.logger.error(f"Failed to load logos for live game: {game.get('id')}") # Changed log prefix
# Draw placeholder text if logos fail
draw_final = ImageDraw.Draw(main_img.convert('RGB'))
self._draw_text_with_outline(draw_final, "Logo Error", (5,5), self.fonts['status'])
self.display_manager.image.paste(main_img.convert('RGB'), (0, 0))
self.display_manager.update_display()
return
center_y = self.display_height // 2
# Draw logos (shifted slightly more inward than NHL perhaps)
home_x = self.display_width - home_logo.width + 10 #adjusted from 18 # Adjust position as needed
home_y = center_y - (home_logo.height // 2)
main_img.paste(home_logo, (home_x, home_y), home_logo)
away_x = -10 #adjusted from 18 # Adjust position as needed
away_y = center_y - (away_logo.height // 2)
main_img.paste(away_logo, (away_x, away_y), away_logo)
# --- Draw Text Elements on Overlay ---
# Note: Rankings are now handled in the records/rankings section below
# Scores (centered, slightly above bottom)
home_score = str(game.get("home_score", "0"))
away_score = str(game.get("away_score", "0"))
score_text = f"{away_score}-{home_score}"
score_width = draw_overlay.textlength(score_text, font=self.fonts['score'])
score_x = (self.display_width - score_width) // 2
score_y = (self.display_height // 2) - 3 #centered #from 14 # Position score higher
self._draw_text_with_outline(draw_overlay, score_text, (score_x, score_y), self.fonts['score'])
# Period/Quarter and Clock (Top center)
period_clock_text = f"{game.get('period_text', '')} {game.get('clock', '')}".strip()
if game.get("is_halftime"): \
period_clock_text = "Halftime" # Override for halftime
elif game.get("is_period_break"):
period_clock_text = game.get("status_text", "Period Break")
status_width = draw_overlay.textlength(period_clock_text, font=self.fonts['time'])
status_x = (self.display_width - status_width) // 2
status_y = 1 # Position at top
self._draw_text_with_outline(draw_overlay, period_clock_text, (status_x, status_y), self.fonts['time'])
# Down & Distance or Scoring Event (Below Period/Clock)
scoring_event = game.get("scoring_event", "")
down_distance = game.get("down_distance_text", "")
if self.display_width > 128:
down_distance = game.get("down_distance_text_long", "")
# Show scoring event if detected, otherwise show down & distance
if scoring_event and game.get("is_live"):
# Display scoring event with special formatting
event_width = draw_overlay.textlength(scoring_event, font=self.fonts['detail'])
event_x = (self.display_width - event_width) // 2
event_y = (self.display_height) - 7
# Color coding for different scoring events
if scoring_event == "TOUCHDOWN":
event_color = (255, 215, 0) # Gold
elif scoring_event == "FIELD GOAL":
event_color = (0, 255, 0) # Green
elif scoring_event == "PAT":
event_color = (255, 165, 0) # Orange
else:
event_color = (255, 255, 255) # White
self._draw_text_with_outline(draw_overlay, scoring_event, (event_x, event_y), self.fonts['detail'], fill=event_color)
elif down_distance and game.get("is_live"): # Only show if live and available
dd_width = draw_overlay.textlength(down_distance, font=self.fonts['detail'])
dd_x = (self.display_width - dd_width) // 2
dd_y = (self.display_height)- 7 # Top of D&D text
down_color = (200, 200, 0) if not game.get("is_redzone", False) else (255,0,0) # Yellowish text
self._draw_text_with_outline(draw_overlay, down_distance, (dd_x, dd_y), self.fonts['detail'], fill=down_color)
# Possession Indicator (small football icon)
possession = game.get("possession_indicator")
if possession: # Only draw if possession is known
ball_radius_x = 3 # Wider for football shape
ball_radius_y = 2 # Shorter for football shape
ball_color = (139, 69, 19) # Brown color for the football
lace_color = (255, 255, 255) # White for laces
# Approximate height of the detail font (4x6 font at size 6 is roughly 6px tall)
detail_font_height_approx = 6
ball_y_center = dd_y + (detail_font_height_approx // 2) # Center ball vertically with D&D text
possession_ball_padding = 3 # Pixels between D&D text and ball
if possession == "away":
# Position ball to the left of D&D text
ball_x_center = dd_x - possession_ball_padding - ball_radius_x
elif possession == "home":
# Position ball to the right of D&D text
ball_x_center = dd_x + dd_width + possession_ball_padding + ball_radius_x
else:
ball_x_center = 0 # Should not happen / no indicator
if ball_x_center > 0: # Draw if position is valid
# Draw the football shape (ellipse)
draw_overlay.ellipse(
(ball_x_center - ball_radius_x, ball_y_center - ball_radius_y, # x0, y0
ball_x_center + ball_radius_x, ball_y_center + ball_radius_y), # x1, y1
fill=ball_color, outline=(0,0,0)
)
# Draw a simple horizontal lace
draw_overlay.line(
(ball_x_center - 1, ball_y_center, ball_x_center + 1, ball_y_center),
fill=lace_color, width=1
)
# Timeouts (Bottom corners) - 3 small bars per team
timeout_bar_width = 4
timeout_bar_height = 2
timeout_spacing = 1
timeout_y = self.display_height - timeout_bar_height - 1 # Bottom edge
# Away Timeouts (Bottom Left)
away_timeouts_remaining = game.get("away_timeouts", 0)
for i in range(3):
to_x = 2 + i * (timeout_bar_width + timeout_spacing)
color = (255, 255, 255) if i < away_timeouts_remaining else (80, 80, 80) # White if available, gray if used
draw_overlay.rectangle([to_x, timeout_y, to_x + timeout_bar_width, timeout_y + timeout_bar_height], fill=color, outline=(0,0,0))
# Home Timeouts (Bottom Right)
home_timeouts_remaining = game.get("home_timeouts", 0)
for i in range(3):
to_x = self.display_width - 2 - timeout_bar_width - (2-i) * (timeout_bar_width + timeout_spacing)
color = (255, 255, 255) if i < home_timeouts_remaining else (80, 80, 80) # White if available, gray if used
draw_overlay.rectangle([to_x, timeout_y, to_x + timeout_bar_width, timeout_y + timeout_bar_height], fill=color, outline=(0,0,0))
# Draw odds if available
if 'odds' in game and game['odds']:
self._draw_dynamic_odds(draw_overlay, game['odds'], self.display_width, self.display_height)
# Draw records or rankings if enabled
if self.show_records or self.show_ranking:
record_font = self.fonts.get('detail', ImageFont.load_default())
# Get team abbreviations
away_abbr = game.get('away_abbr', '')
home_abbr = game.get('home_abbr', '')
record_bbox = draw_overlay.textbbox((0,0), "0-0", font=record_font)
record_height = record_bbox[3] - record_bbox[1]
record_y = self.display_height - record_height - 4
self.logger.debug(f"Record positioning: height={record_height}, record_y={record_y}, display_height={self.display_height}")
# Display away team info
if away_abbr:
if self.show_ranking and self.show_records:
# When both rankings and records are enabled, rankings replace records completely
away_rank = self._team_rankings_cache.get(away_abbr, 0)
if away_rank > 0:
away_text = f"#{away_rank}"
else:
# Show nothing for unranked teams when rankings are prioritized
away_text = ''
elif self.show_ranking:
# Show ranking only if available
away_rank = self._team_rankings_cache.get(away_abbr, 0)
if away_rank > 0:
away_text = f"#{away_rank}"
else:
away_text = ''
elif self.show_records:
# Show record only when rankings are disabled
away_text = game.get('away_record', '')
else:
away_text = ''
if away_text:
away_record_x = 3
self.logger.debug(f"Drawing away ranking '{away_text}' at ({away_record_x}, {record_y}) with font size {record_font.size if hasattr(record_font, 'size') else 'unknown'}")
self._draw_text_with_outline(draw_overlay, away_text, (away_record_x, record_y), record_font)
# Display home team info
if home_abbr:
if self.show_ranking and self.show_records:
# When both rankings and records are enabled, rankings replace records completely
home_rank = self._team_rankings_cache.get(home_abbr, 0)
if home_rank > 0:
home_text = f"#{home_rank}"
else:
# Show nothing for unranked teams when rankings are prioritized
home_text = ''
elif self.show_ranking:
# Show ranking only if available
home_rank = self._team_rankings_cache.get(home_abbr, 0)
if home_rank > 0:
home_text = f"#{home_rank}"
else:
home_text = ''
elif self.show_records:
# Show record only when rankings are disabled
home_text = game.get('home_record', '')
else:
home_text = ''
if home_text:
home_record_bbox = draw_overlay.textbbox((0,0), home_text, font=record_font)
home_record_width = home_record_bbox[2] - home_record_bbox[0]
home_record_x = self.display_width - home_record_width - 3
self.logger.debug(f"Drawing home ranking '{home_text}' at ({home_record_x}, {record_y}) with font size {record_font.size if hasattr(record_font, 'size') else 'unknown'}")
self._draw_text_with_outline(draw_overlay, home_text, (home_record_x, record_y), record_font)
# Composite the text overlay onto the main image
main_img = Image.alpha_composite(main_img, overlay)
main_img = main_img.convert('RGB') # Convert for display
# Display the final image
self.display_manager.image.paste(main_img, (0, 0))
self.display_manager.update_display() # Update display here for live
except Exception as e:
self.logger.error(f"Error displaying live Football game: {e}", exc_info=True) # Changed log prefix
+380
View File
@@ -0,0 +1,380 @@
import logging
from typing import Any, Dict, Optional
from PIL import Image, ImageDraw, ImageFont
from src.base_classes.data_sources import ESPNDataSource
from src.base_classes.sports import SportsCore, SportsLive
from src.cache_manager import CacheManager
from src.display_manager import DisplayManager
class Hockey(SportsCore):
"""Base class for hockey sports with common functionality."""
def __init__(
self,
config: Dict[str, Any],
display_manager: DisplayManager,
cache_manager: CacheManager,
logger: logging.Logger,
sport_key: str,
):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
self.data_source = ESPNDataSource(logger)
self.sport = "hockey"
self.show_shots_on_goal = self.mode_config.get("show_shots_on_goal", False)
def _extract_game_details(self, game_event: Dict) -> Optional[Dict]:
"""Extract relevant game details from ESPN NCAA FB API response."""
# --- THIS METHOD MAY NEED ADJUSTMENTS FOR NCAA FB API DIFFERENCES ---
details, home_team, away_team, status, situation = (
self._extract_game_details_common(game_event)
)
if details is None or home_team is None or away_team is None or status is None:
return
try:
competition = game_event["competitions"][0]
status = competition["status"]
powerplay = False
penalties = ""
# A competitor may legitimately arrive without a "statistics"
# array (pre-game feeds, and some in-progress ones). Reading it
# unguarded raised KeyError inside the generator and dropped the
# WHOLE event, discarding valid scores and status. Default to an
# empty list so the saves/shots figures fall back to 0 instead.
home_stats = home_team.get("statistics", [])
away_stats = away_team.get("statistics", [])
home_team_saves = next(
(
int(c["displayValue"])
for c in home_stats
if c.get("name") == "saves"
),
0,
)
home_team_saves_per = next(
(
float(c["displayValue"])
for c in home_stats
if c.get("name") == "savePct"
),
0.0,
)
away_team_saves = next(
(
int(c["displayValue"])
for c in away_stats
if c.get("name") == "saves"
),
0,
)
away_team_saves_per = next(
(
float(c["displayValue"])
for c in away_stats
if c.get("name") == "savePct"
),
0.0,
)
home_shots = 0
away_shots = 0
if home_team_saves_per > 0:
away_shots = round(home_team_saves / home_team_saves_per)
if away_team_saves_per > 0:
home_shots = round(away_team_saves / away_team_saves_per)
if situation and status["type"]["state"] == "in":
# Detect scoring events from status detail
# status_detail = status["type"].get("detail", "")
powerplay = situation.get("isPowerPlay", False)
penalties = situation.get("penalties", "")
# Format period/quarter
period = status.get("period", 0)
period_text = ""
if status["type"]["state"] == "in":
if period == 0:
period_text = "Start" # Before kickoff
elif period >= 1 and period <= 3:
period_text = f"P{period}" # OT starts after Q4
elif period > 3:
period_text = f"OT{period - 3}" # OT starts after Q4
elif status["type"]["state"] == "post":
if period > 3:
period_text = "Final/OT"
else:
period_text = "Final"
elif status["type"]["state"] == "pre":
period_text = details.get("game_time", "") # Show time for upcoming
details.update(
{
"period": period,
"period_text": period_text, # Formatted quarter/status
"clock": status.get("displayClock", "0:00"),
"power_play": powerplay,
"penalties": penalties,
"home_shots": home_shots,
"away_shots": away_shots,
}
)
# Basic validation (can be expanded)
if not details["home_abbr"] or not details["away_abbr"]:
self.logger.warning(
f"Missing team abbreviation in event: {details['id']}"
)
return None
self.logger.debug(
f"Extracted: {details['away_abbr']}@{details['home_abbr']}, Status: {status['type']['name']}, Live: {details['is_live']}, Final: {details['is_final']}, Upcoming: {details['is_upcoming']}"
)
return details
except Exception as e:
# Log the problematic event structure if possible
self.logger.error(
f"Error extracting game details: {e} from event: {game_event.get('id')}",
exc_info=True,
)
return None
class HockeyLive(Hockey, SportsLive):
def __init__(
self,
config: Dict[str, Any],
display_manager: DisplayManager,
cache_manager: CacheManager,
logger: logging.Logger,
sport_key: str,
):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
def _test_mode_update(self):
if self.current_game and self.current_game["is_live"]:
# For testing, we'll just update the clock to show it's working
minutes = int(self.current_game["clock"].split(":")[0])
seconds = int(self.current_game["clock"].split(":")[1])
seconds -= 1
if seconds < 0:
seconds = 59
minutes -= 1
if minutes < 0:
minutes = 19
if self.current_game["period"] < 3:
self.current_game["period"] += 1
else:
self.current_game["period"] = 1
self.current_game["clock"] = f"{minutes:02d}:{seconds:02d}"
# Always update display in test mode
def _draw_scorebug_layout(self, game: Dict, force_clear: bool = False) -> None:
"""Draw the detailed scorebug layout for a live NCAA FB game.""" # Updated docstring
try:
main_img = Image.new(
"RGBA", (self.display_width, self.display_height), (0, 0, 0, 255)
)
overlay = Image.new(
"RGBA", (self.display_width, self.display_height), (0, 0, 0, 0)
)
draw_overlay = ImageDraw.Draw(
overlay
) # Draw text elements on overlay first
home_logo = self._load_and_resize_logo(
game["home_id"],
game["home_abbr"],
game["home_logo_path"],
game.get("home_logo_url"),
)
away_logo = self._load_and_resize_logo(
game["away_id"],
game["away_abbr"],
game["away_logo_path"],
game.get("away_logo_url"),
)
if not home_logo or not away_logo:
self.logger.error(
f"Failed to load logos for live game: {game.get('id')}"
) # Changed log prefix
# Draw placeholder text if logos fail
draw_final = ImageDraw.Draw(main_img.convert("RGB"))
self._draw_text_with_outline(
draw_final, "Logo Error", (5, 5), self.fonts["status"]
)
self.display_manager.image.paste(main_img.convert("RGB"), (0, 0))
self.display_manager.update_display()
return
center_y = self.display_height // 2
# Draw logos (shifted slightly more inward than NHL perhaps)
home_x = (
self.display_width - home_logo.width + 10
) # adjusted from 18 # Adjust position as needed
home_y = center_y - (home_logo.height // 2)
main_img.paste(home_logo, (home_x, home_y), home_logo)
away_x = -10 # adjusted from 18 # Adjust position as needed
away_y = center_y - (away_logo.height // 2)
main_img.paste(away_logo, (away_x, away_y), away_logo)
# --- Draw Text Elements on Overlay ---
# Note: Rankings are now handled in the records/rankings section below
# Period/Quarter and Clock (Top center)
period_clock_text = (
f"{game.get('period_text', '')} {game.get('clock', '')}".strip()
)
if game.get("is_period_break"):
period_clock_text = game.get("status_text", "Period Break")
status_width = draw_overlay.textlength(
period_clock_text, font=self.fonts["time"]
)
status_x = (self.display_width - status_width) // 2
status_y = 1 # Position at top
self._draw_text_with_outline(
draw_overlay,
period_clock_text,
(status_x, status_y),
self.fonts["time"],
)
# Scores (centered, slightly above bottom)
home_score = str(game.get("home_score", "0"))
away_score = str(game.get("away_score", "0"))
score_text = f"{away_score}-{home_score}"
score_width = draw_overlay.textlength(score_text, font=self.fonts["score"])
score_x = (self.display_width - score_width) // 2
score_y = (
self.display_height // 2
) - 3 # centered #from 14 # Position score higher
self._draw_text_with_outline(
draw_overlay, score_text, (score_x, score_y), self.fonts["score"]
)
# Shots on Goal
if self.show_shots_on_goal:
shots_font = self.fonts.get('detail', ImageFont.load_default())
home_shots = str(game.get("home_shots", "0"))
away_shots = str(game.get("away_shots", "0"))
shots_text = f"{away_shots} SHOTS {home_shots}"
shots_bbox = draw_overlay.textbbox((0, 0), shots_text, font=shots_font)
shots_height = shots_bbox[3] - shots_bbox[1]
shots_y = self.display_height - shots_height - 1
shots_width = draw_overlay.textlength(shots_text, font=shots_font)
shots_x = (self.display_width - shots_width) // 2
self._draw_text_with_outline(
draw_overlay, shots_text, (shots_x, shots_y), shots_font
)
# Draw odds if available
if "odds" in game and game["odds"]:
self._draw_dynamic_odds(
draw_overlay, game["odds"], self.display_width, self.display_height
)
# Draw records or rankings if enabled
if self.show_records or self.show_ranking:
record_font = self.fonts.get('detail', ImageFont.load_default())
# Get team abbreviations
away_abbr = game.get("away_abbr", "")
home_abbr = game.get("home_abbr", "")
record_bbox = draw_overlay.textbbox((0, 0), "0-0", font=record_font)
record_height = record_bbox[3] - record_bbox[1]
record_y = self.display_height - record_height - 1
self.logger.debug(
f"Record positioning: height={record_height}, record_y={record_y}, display_height={self.display_height}"
)
# Display away team info
if away_abbr:
if self.show_ranking and self.show_records:
# When both rankings and records are enabled, rankings replace records completely
away_rank = self._team_rankings_cache.get(away_abbr, 0)
if away_rank > 0:
away_text = f"#{away_rank}"
else:
# Show nothing for unranked teams when rankings are prioritized
away_text = ""
elif self.show_ranking:
# Show ranking only if available
away_rank = self._team_rankings_cache.get(away_abbr, 0)
if away_rank > 0:
away_text = f"#{away_rank}"
else:
away_text = ""
elif self.show_records:
# Show record only when rankings are disabled
away_text = game.get("away_record", "")
else:
away_text = ""
if away_text:
away_record_x = 3
self.logger.debug(
f"Drawing away ranking '{away_text}' at ({away_record_x}, {record_y}) with font size {record_font.size if hasattr(record_font, 'size') else 'unknown'}"
)
self._draw_text_with_outline(
draw_overlay,
away_text,
(away_record_x, record_y),
record_font,
)
# Display home team info
if home_abbr:
if self.show_ranking and self.show_records:
# When both rankings and records are enabled, rankings replace records completely
home_rank = self._team_rankings_cache.get(home_abbr, 0)
if home_rank > 0:
home_text = f"#{home_rank}"
else:
# Show nothing for unranked teams when rankings are prioritized
home_text = ""
elif self.show_ranking:
# Show ranking only if available
home_rank = self._team_rankings_cache.get(home_abbr, 0)
if home_rank > 0:
home_text = f"#{home_rank}"
else:
home_text = ""
elif self.show_records:
# Show record only when rankings are disabled
home_text = game.get("home_record", "")
else:
home_text = ""
if home_text:
home_record_bbox = draw_overlay.textbbox(
(0, 0), home_text, font=record_font
)
home_record_width = home_record_bbox[2] - home_record_bbox[0]
home_record_x = self.display_width - home_record_width - 3
self.logger.debug(
f"Drawing home ranking '{home_text}' at ({home_record_x}, {record_y}) with font size {record_font.size if hasattr(record_font, 'size') else 'unknown'}"
)
self._draw_text_with_outline(
draw_overlay,
home_text,
(home_record_x, record_y),
record_font,
)
# Composite the text overlay onto the main image
main_img = Image.alpha_composite(main_img, overlay)
main_img = main_img.convert("RGB") # Convert for display
# Display the final image
self.display_manager.image.paste(main_img, (0, 0))
self.display_manager.update_display() # Update display here for live
except Exception as e:
self.logger.error(
f"Error displaying live Hockey game: {e}", exc_info=True
) # Changed log prefix
+17
View File
@@ -0,0 +1,17 @@
"""Sports scoreboard base classes.
Formerly the single module ``src/base_classes/sports.py``; now a package so
capabilities can be composed instead of accumulating in one class. See
docs/SPORTS_UNIFICATION.md for the architecture. The import path is
unchanged: ``from src.base_classes.sports import SportsCore`` still works.
"""
from .core import SportsCore
from .modes import SportsLive, SportsRecent, SportsUpcoming
__all__ = [
"SportsCore",
"SportsUpcoming",
"SportsRecent",
"SportsLive",
]

Some files were not shown because too many files have changed in this diff Show More