Compare commits

...
4 Commits
Author SHA1 Message Date
ChuckandClaude Opus 5.5 a5ec645d25 refactor(display): run() stage 2 - an Arbiter decides the scheduled-off blank, follower and WiFi notice, no behaviour change (#733)
run() stage 2: a pure Arbiter.decide() (src/display_arbiter.py) chooses scheduled-off, follower and WiFi notices; everything else takes the existing path. Golden traces byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 22:30:52 -04:00
ChuckandClaude Opus 5.5 084697346b feat(fetch): one ESPN scoreboard cache key and a max-age response cache (fetch service stage 2) (#728)
Fetch service stage 2: one ESPN scoreboard cache key shared across the sports base classes (legacy keys still read), and a max-age response cache in the fetch service.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 18:15:28 -04:00
ChuckandClaude Opus 5.5 a7b3f33952 feat(web): Rotation, Operation History, Config Editor and Backup & Restore become ES-module pages (stage 2) (#727)
Rotation, Operation History, Config Editor and Backup & Restore become ES-module pages (stage 2): no inline scripts or onclick in the four partials, delegated data-action listeners, reads cancelled on swap-out, old globals kept as deprecated aliases through window.LEDMatrix, and four new DOM suites.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 18:02:03 -04:00
ChuckandClaude Opus 5.5 c14002edc3 fix(status): runtime status agrees with the heartbeat; current-status republishes on wake (#726)
The runtime status snapshot now agrees with the display heartbeat, and current-status is republished when the display wakes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 17:47:02 -04:00
42 changed files with 4464 additions and 928 deletions
+97
View File
@@ -19,6 +19,29 @@ accepts both, but the store flags the old spelling as deprecated
## Unreleased
### Web UI: four more tabs are ES-module pages (stage 2)
- Rotation, Operation History, Config Editor and Backup & Restore follow the
Cache tab (#703): each partial's inline `<script>` is now
`static/v3/js/pages/<name>.js` (`durations`, `operation-history`,
`raw-json`, `backup-restore`), started once per swap-in by the page
registry and stopped on swap-out. None of the four partials has an inline
script or `onclick` any more. Buttons carry `data-action` and use one
delegated listener. Server data is drawn with `textContent`.
- Old globals keep working as deprecated aliases through `window.LEDMatrix`
(one console warning each): `formatJson`, `manualValidateJson`,
`validateJSON`, `saveMainConfig`, `saveSecretsConfig`, `exportBackup`,
`loadBackupList`, `validateRestoreFile`, `clearRestore`, `runRestore`.
- Reads are cancelled when a tab is swapped away. Writes (save, delete,
export, restore) are not, and their result is still reported.
- `core/api.js` accepts a raw `body` (a `FormData` upload).
`PluginOrderList.init()` accepts a `signal` for its plugin-list request.
- Small fixes on the way: the Config Editor's "Invalid JSON" line no longer
puts the parser's message into `innerHTML`, and Operation History's
"Showing x to y of z" now resets when nothing matches.
- New DOM suites `test/js/dom/test_{durations,operation_history,raw_json,backup_restore}_page.js`.
`test/web_interface/test_es_modules.py` pins the converted pages and the aliases.
### Garbage-collection pauses in the frame stats
- The display now times every Python garbage collection
@@ -96,6 +119,50 @@ policies are unchanged.
`GET /api/v3/plugins/fetch-stats`.
- `fetch_service` is a core config section (`src/core_config_keys.py`).
### Shared fetch service (stage 2: one scoreboard cache key, a max-age response cache)
- **One cache key per ESPN scoreboard.** `espn_scoreboard_cache_key(sport,
league, dates)` in `src/common/espn_dates.py` names a scoreboard by ESPN's
own path (`football`/`nfl`) and its `dates=` value, so every consumer of
the same scoreboard shares one cached copy. Before, odds-ticker cached it as
`scoreboard_data_{sport}_{league}_{date}`, `APIHelper` as
`espn_{sport}_{league}_{date}` and the scoreboards as
`{sport_key}_schedule_{window}`.
- **Cache-through helpers.** `get_espn_scoreboard()` returns a cached copy at
most `max_age` seconds old and otherwise fetches with
`fetch_espn_scoreboard` and caches the result; `read_espn_scoreboard_cache()`
and `store_espn_scoreboard_cache()` are the two halves (the read takes an
`accept(data, age)` predicate, e.g. a shorter limit for a payload holding a
live game). A read checks the
record's own timestamp, so a writer's stored ttl can no longer make a
reader take data older than its own TTL; the shared entry stores no ttl.
Old keys are passed as `legacy_keys` and read after the canonical one for
one release, so an upgrade does not refetch every league at once.
- **Core callers use the key.** `APIHelper.fetch_espn_scoreboard` caches
under it by default (an explicit `cache_key` still works as before; the old
default key is read as a fallback). `SportsFetchMixin` gains
`_schedule_cache_key()` and `_cached_schedule()` for the scoreboards'
schedule windows (the same read as before, with the old key as fallback,
and a miss deletes the previous day's copy of a sliding window), and
`_fetch_season_directly(cache_key=None)` uses the canonical key. The
scoreboards and odds-ticker move to it in a plugins release that requires
this core.
- **Response cache.** A `200` with `Cache-Control: max-age=N` (minus `Age`)
answers an identical GET for N seconds without a request; ESPN sends no
validators, only max-age (1 to ~500 s, measured 2026-10-02). A caller says
how old a response it accepts with `fetch_get(..., cache_max_age=)` /
`fetch_espn_scoreboard(..., cache_max_age=)`; one that does not say gets at
most 30 s (`fetch_service.response_cache.default_max_age`).
`BaseOddsManager.get_odds` passes its update interval, `APIHelper.get` its
`cache_ttl`, and `get_espn_scoreboard` its `max_age`. `no-store`,
`no-cache`, `private`, `Vary: *` and `Set-Cookie` responses are never kept.
Bounded: 64 entries, 6 MB, 2 MB each; never longer than 10 minutes.
- **Counters.** `memo_hits` (answered by the response cache), `cache_hits`
(scoreboard fetches answered by a shared cache entry) and
`legacy_cache_hits` (reads from a pre-stage-2 key), per plugin and per
host, and the response cache's size, in `GET /api/v3/plugins/fetch-stats`.
- No new module; every change is additive to existing signatures.
### Control socket (stage 2: wake-ups, brightness, plugin reload)
- **Socket commands land at once.** Stage 1's socket was no faster than the
@@ -141,6 +208,9 @@ policies are unchanged.
this release: plugins reach it through `APIHelper` and `espn_dates`, and
should not import it directly until a plugin-facing API ships (stage 3), so
it sets no `ledmatrix_min_version` floor.
- `src/display_arbiter.py` -- the display loop's Arbiter (see Tooling).
Core-internal: plugins have no reason to import it, so it sets no
`ledmatrix_min_version` floor.
### Tooling
@@ -158,6 +228,17 @@ policies are unchanged.
(`_dispatch_first_frame`, `_resolve_durations`, `_resolve_active_mode`,
`_needs_high_fps`, `_advance_after_screen` and others), and the traces are
identical before and after the move.
- Display loop stage 2: an Arbiter decides who gets the panel. Each pass,
`run()` gathers a small snapshot (the schedule, the on-demand flag, the
sync follower, the pending WiFi notice) and calls
`Arbiter.decide(state, inputs, now)` in `src/display_arbiter.py`, a pure
function, which returns a `ScreenPlan`. It decides the scheduled-off
blank, the follower frame and the WiFi notice; on-demand, live priority,
Vegas and the rotation return a `LEGACY` plan and run the existing code.
The WiFi notice's mid-screen rule (`wifi_notice_preempts`) moves there
too. No behaviour change: the golden traces regenerate byte-identical.
`test/test_display_arbiter.py` tests `decide()` with a table of all 16
combinations of its inputs.
### Fixes
@@ -223,6 +304,22 @@ policies are unchanged.
being stopped, blanks the panel within about a second. It used to stay on
until the next minute, because the once-a-minute schedule check had
already run that minute and the session had overridden its answer.
- `/api/v3/plugins/installed` no longer reports the display's plugins as
`live` while `/api/v3/health` says `display_loop: stalled`. The runtime
snapshot is written from its own thread, which kept going while the render
loop was hung. The web interface now also reads the render loop's
heartbeat: a fresh snapshot whose process's heartbeat is 60 s or older is
`data.runtime.status: "stalled"`, with the per-plugin fields null, and
`data.runtime` gains `heartbeat_age_seconds`. A snapshot from a process
that no longer exists, as after a watchdog kill (systemd removes the
heartbeat when the service stops), is `stale` at once instead of `live`
for up to 180 s. No new files or writes: both checks are on the reading
side.
- `/api/v3/display/current-status` reflects a wake from scheduled-off, a
schedule-off blank, or an on-demand session starting or ending at once,
even when the mode name stays the same. The display republished its
current state only on a mode change or every 30 s, so `is_display_active`
and `on_demand_active` could be up to 30 s out of date.
### Scrolling
+42 -2
View File
@@ -1054,8 +1054,16 @@ values, exceptions and retries are what they were.
next identical request revalidates, and a `304 Not Modified` comes back to
your code as the original `200` with its body. ESPN currently sends
neither, so this does nothing there.
- **Response cache.** A response whose server says `Cache-Control:
max-age=N` answers an identical GET for those N seconds without a
request (ESPN sends 1 to ~500 s). It never hands you a response older
than you accept: pass `cache_max_age=<your TTL>` to `fetch_get()` or
`fetch_espn_scoreboard()` (0 always asks the network); without it a
response is reused for at most 30 seconds.
- **Counters.** Requests, merged requests, bytes, 304s, errors and time spent
waiting are counted per plugin and per host, and published for the web UI
waiting, and requests answered without the network (`memo_hits` from the
response cache, `cache_hits` from a shared scoreboard cache entry), are
counted per plugin and per host, and published for the web UI
at `GET /api/v3/plugins/fetch-stats` (see
[REST_API_REFERENCE.md](REST_API_REFERENCE.md#get-fetch-statistics)). A
request is counted against your plugin when it runs inside your
@@ -1066,6 +1074,36 @@ What is not covered yet: requests a plugin makes with its own `requests.get()`
or `Session.get()` calls. They work as before but are invisible to the
budgets and counters.
### One cache key per ESPN scoreboard
Cache an ESPN scoreboard under `espn_scoreboard_cache_key(sport, league,
dates)` (`src.common.espn_dates`), not a key of your own, so every plugin
showing that league shares one fetch and one cached copy. `sport` and
`league` are ESPN's path segments (`football`, `college-football`), and
`dates` is what you send as `dates=` (`"20261004"`, `"202610"`,
`"20260925-20261016"`, a `date`, or `None` for the undated scoreboard).
```python
from src.common.espn_dates import get_espn_scoreboard
data = get_espn_scoreboard(
self.session, "football", "nfl", "20261004",
cache_manager=self.cache_manager,
max_age=300, # your TTL: nothing older comes back
legacy_keys=["my_old_key_20261004"], # read once while upgrading
)
```
`get_espn_scoreboard` returns a cached copy at most `max_age` seconds old,
whoever wrote it, and otherwise fetches with `fetch_espn_scoreboard`
(`limit=500`, ranges split the way ESPN requires) and caches the result
without a ttl, so each reader applies its own age limit. `max_age=0` always
fetches but still leaves the copy for others. For a two-step read, use
`read_espn_scoreboard_cache()` and `store_espn_scoreboard_cache()` around
your own fetch. Scoreboards built on `SportsFetchMixin` get
`_schedule_cache_key(datestring)` and `_cached_schedule(key, legacy_keys)`
for their schedule windows. All of this is in the core release after 3.8.0.
The settings live in `config.json` under `fetch_service`, read when the
display starts and on a config reload:
@@ -1084,7 +1122,9 @@ display starts and on a config reload:
the bare domain); `"per_second": 0` removes a budget. `"enabled": false`
turns the whole service into a plain `session.get()`. Two further switches,
`"single_flight": false` and `"conditional_get": false`, turn off merging and
revalidation.
revalidation. `"response_cache": {"enabled": false}` turns off the response
cache; its `default_max_age` (30) is the limit for callers that pass no
`cache_max_age`.
---
+26 -7
View File
@@ -551,7 +551,8 @@ List all installed plugins with their status and metadata.
"status": "live",
"published_at": 1790000030.0,
"age_seconds": 12.4,
"stale_after": 180.0
"stale_after": 180.0,
"heartbeat_age_seconds": 2.1
}
}
}
@@ -572,10 +573,15 @@ until the display restarts. A plugin a live snapshot does not list is
`loaded: false`, `state: "unloaded"`.
`runtime.status` says whether to believe them: `live` (fresh snapshot from
a running display), `stale` (not refreshed within `stale_after` seconds: the
display is hung or died), `stopped` (the display shut down) or `unknown`
a running display), `stalled` (fresh snapshot, but the same process's
render-loop heartbeat is 60 s or older -- the render loop is hung, as
[`/health`](#health-check)'s `display_loop: stalled` says), `stale` (not refreshed
within `stale_after` seconds, or the process that wrote it no longer exists:
the display is hung or died), `stopped` (the display shut down) or `unknown`
(nothing published yet). Unless it is `live`, every one of those fields is
`null`. Health and metrics are at [`/plugins/health`](#get-plugin-health)
`null`. `heartbeat_age_seconds` is the heartbeat's age when it was taken into
account, `null` otherwise (no heartbeat, as on the dev server, or one from
another process). Health and metrics are at [`/plugins/health`](#get-plugin-health)
and `/plugins/metrics`.
`vegas_participation` is what Vegas mode does with the plugin: `"scroll"`,
@@ -1042,7 +1048,8 @@ or `unknown` (nothing published; `data.data` is `null`).
"totals": {"requests": 412, "merged": 3, "not_modified": 0,
"errors": 1, "http_errors": 2, "retries": 0,
"throttled": 0, "overruns": 0, "bytes": 18234011,
"wait_seconds": 0.0},
"wait_seconds": 0.0, "memo_hits": 21, "cache_hits": 40,
"legacy_cache_hits": 2},
"plugins": {
"football-scoreboard": {"requests": 240, "merged": 2, "bytes": 9120330,
"hosts": {"site.api.espn.com": 180,
@@ -1053,9 +1060,11 @@ or `unknown` (nothing published; `data.data` is `null`).
"site.api.espn.com": {"requests": 301, "...": "as in totals"}
},
"validators": {"entries": 0, "bytes": 0},
"response_cache": {"entries": 3, "bytes": 412004},
"config": {"enabled": true, "single_flight": true,
"conditional_get": true, "max_wait_seconds": 2.0,
"rate_limits": {"*.espn.com": {"per_second": 20.0, "burst": 200.0}}}
"rate_limits": {"*.espn.com": {"per_second": 20.0, "burst": 200.0}},
"response_cache": true, "default_max_age": 30.0}
}
}
}
@@ -1067,6 +1076,16 @@ flight, `not_modified` 304s served from the stored body, `errors` transport
failures and `http_errors` responses with status 400 or above. `bytes` is the
decoded body size. `core` is everything no plugin made.
Three counters are requests that never reached the network: `memo_hits`
were answered from the short response cache (a response still inside the
`Cache-Control: max-age` its server gave it), and `cache_hits` were
scoreboard fetches answered from a shared ESPN scoreboard cache entry
(`espn_scoreboard_cache_key`). `legacy_cache_hits` counts reads served from a
key that predates the shared one; it should fall to zero within a day of an
upgrade. A plugin's `hosts` counts are requests plus merged requests,
`memo_hits` and `cache_hits`: everything it asked for.
`response_cache` is the size of the response cache now.
### Get/Set Plugin Limits
**GET** `/api/v3/plugins/limits/<plugin_id>`
@@ -1138,7 +1157,7 @@ it is neither installed nor configured).
"last_updated": "2025-01-15T10:30:00"
}
},
"runtime": {"status": "live", "published_at": 1790000030.0, "age_seconds": 12.4, "stale_after": 180.0}
"runtime": {"status": "live", "published_at": 1790000030.0, "age_seconds": 12.4, "stale_after": 180.0, "heartbeat_age_seconds": 2.1}
}
```
+64 -5
View File
@@ -44,7 +44,9 @@ Each pass, in order:
3. Poll on-demand requests and expiry, release plugins loaded only for
on-demand, tick plugin updates, drop an expired WiFi notice, evaluate
the schedule (an on-demand session overrides scheduled-off), apply the
brightness target.
brightness target. Then gather the Arbiter's inputs
(`_arbiter_inputs`) and call `Arbiter.decide()`, which picks one of
steps 4-6 or returns `LEGACY` for steps 7-9 (stage 2).
4. **Scheduled off:** blank, dwell up to 60 s. `_blank_while_scheduled_off`
5. **Follower:** render one frame from the leader. `_run_follower_frame`
6. **WiFi notice** (unless on-demand): draw it, dwell 0.5 s. `_show_wifi_notice`.
@@ -70,8 +72,9 @@ Each pass, in order:
next mode (`_advance_after_screen`).
The helpers named above were extracted in stage 1 without changing
behaviour. The frame loops, the Vegas branch and every early exit are still
inline in `run()`.
behaviour. Since stage 2 the choice between steps 4, 5, 6 and the rest is
made by `Arbiter.decide()` in `src/display_arbiter.py`. The frame loops, the
Vegas branch and every early exit are still inline in `run()`.
## Target design
@@ -160,7 +163,7 @@ that the harness patches in today.
| 4 | Vegas as a Source driven by `run_frame()` | none intended | traces against the real coordinator; ledpi Vegas soak A/B |
| 5 | Plugins declare `frame_policy` | DEBUG instead of INFO for the FPS line | traces; soak on a static-heavy rotation |
### Stage 1 (this PR)
### Stage 1 (#704)
- `test/_run_loop_harness.py` builds a real `DisplayController` through
`__init__` on in-memory fakes (plugins, cache, config service, plugin
@@ -194,7 +197,7 @@ that the harness patches in today.
- Twelve helpers were extracted from `run()` (listed under "What `run()`
does today"). Breaking any one of them fails at least one golden trace.
### Stage 2: Arbiter, starting with Follower and Wifi
### Stage 2: Arbiter, starting with Follower and Wifi (done)
1. Add `ScreenPlan` and an `Arbiter` with the ScheduledOff gate, Follower
and Wifi. Every other case returns a `LEGACY` plan, which means "carry on
@@ -211,6 +214,41 @@ that the harness patches in today.
Follower and Wifi go first because each is one self-contained branch that
ends the pass. They prove the plumbing without touching the frame loops.
What shipped:
- `src/display_arbiter.py` (on the mypy ratchet) holds `Source`
(`SCHEDULED_OFF`, `FOLLOWER`, `WIFI`, `LEGACY`), `ArbiterInputs`,
`ArbiterState`, `WifiNotice`, `ScreenPlan` and `Arbiter.decide`.
`ScreenPlan` has only the fields stage 2 uses: `source`, `max_duration`
(60 s for the blank, 0.5 s for the notice, the constants `run()` used to
hard-code) and `notice`. `mode`, `plugin`, the other durations,
`frame_policy` and `preemptible_by` arrive with the Sources that need them.
- `ArbiterState` is empty: no stage-2 Source remembers anything between
passes. `now` is passed but not read, because the top-of-pass WiFi check
never compared the expiry and must not start (the table pins this).
- `ArbiterInputs` holds `schedule_on`, `on_demand_active`,
`follower_active` and `wifi_notice`. `_arbiter_inputs` derives
`schedule_on` as `is_display_active and not on_demand_schedule_override`,
so the gate (blank when the schedule is off and no on-demand session
overrides it) blanks exactly when `is_display_active` is False, as before,
including #714's on-demand ending in off hours. It reads the WiFi notice
only when the notice could win, because `_check_wifi_status_message` has
side effects (its 1 Hz throttle, deleting an expired file) that those
passes never had.
- The mid-screen rule is `wifi_notice_preempts(notice, on_demand, now)`,
which `_wifi_notice_pending` calls; it does compare the expiry.
- `run()` still calls `_publish_current_mode_state_if_changed`,
`_apply_pending_vegas_init` and `process_deferred_updates` at the same
points relative to the branches, so the order of side effects in a pass
is unchanged.
- `test/test_display_arbiter.py`: the 16-row table (every combination of
the four inputs, written out), the mid-screen table, purity checks (no
clock reads, nothing mutated, no I/O imports), and the controller's
snapshot through an on-demand session that overrides the schedule and
ends. A mutation run broke 23 pieces once each (the gate, the order, each
Source, the dwells, the expiry comparison, the snapshot's reads, each
dispatch in `run()`); every one failed a test.
### Stage 3: ScreenRunner and `PREEMPTED`
Move the two frame loops, the make-up dwell and the dynamic-duration exit
@@ -218,6 +256,27 @@ into `ScreenRunner.run(plan)` with an injected `FrameClock`. Replace the
five re-checks with `PREEMPTED`. Add the OnDemand, Live and Rotation Sources
so `LEGACY` is left meaning only Vegas.
Concretely, from where stage 2 left off:
1. `ArbiterState` gains the rotation index, the on-demand mode list, index,
expiry and pin, and the live resume point (today `current_mode_index`,
`on_demand_*` and the live-priority stash). `ArbiterInputs` gains the
live modes (`_collect_live_modes`) and whether Vegas is enabled and keeps
live content in the ticker.
2. OnDemand returns its current mode with `_clamp_to_on_demand`'s bound,
reading `now` for the expiry. Live returns the next live mode
(round-robin). Rotation returns `available_modes[current_mode_index]`.
`ScreenPlan` gains `mode`, `plugin`, `min_duration`, `max_duration`,
`dynamic`, `frame_policy` and `preemptible_by`.
3. `ScreenRunner.run(plan)` returns an `ExitReason`; `state.after(plan,
outcome)` replaces `_advance_after_screen` and the live-resume
bookkeeping. Each mid-screen check asks `decide()` whether a Source in
`plan.preemptible_by` now wins, so `_screen_preempted`,
`_check_live_takeover` and `_wifi_notice_pending` become one call.
4. The control socket (`_drain_control_commands`, `_wait_for_control`) and
state publishing stay where they are; the runner calls them at its
service points.
This stage touches frame pacing (the 8 ms deadline sleep, the 1 ms yield),
so it needs a frame soak on ledpi, A/B against main. Coordinate with
whoever owns scroll performance (`docs/SCROLL_PERFORMANCE.md`).
+33 -8
View File
@@ -46,6 +46,7 @@ static/v3/js/
store.js (the one installed-plugin store), form/renderer.js
pages/ one module per tab partial
cache.js export init(root, ctx), destroy(root, ctx)
durations.js, operation-history.js, raw-json.js, backup-restore.js
...
```
@@ -57,9 +58,29 @@ A converted partial has no `<script>`. Its root element names its page:
<div class="..." data-page="cache"> ... </div>
```
`core/boot.js` registers each page with a loader:
`registry.register('cache', () => import('../pages/cache.js'))`. A page's
module is fetched only when its partial first appears.
`core/boot.js` lists each page with a loader,
`'cache': page(function() { return import('../pages/cache.js'); })`, and
registers them all. A page's module is fetched only when its partial first
appears. `page()` remembers the module once loaded, so the alias of an old
synchronous global (`validateJSON` returns a boolean) still answers
synchronously while its page is on screen.
The conventions the converted pages share:
- **Buttons name an action.** A partial's buttons carry `data-action` (and
any argument as another `data-*` attribute) instead of an `onclick` that
names a global. One delegated listener on the page root handles them all,
including rows drawn later.
- **Server data is drawn with `textContent`**, never a markup string.
- **Reads are cancelled, writes are not.** Loads pass `ctx.signal`, so a swap
cancels them. Saves, deletes, exports and restores do not: the server
finishes them anyway, so the page still reports the result in a
notification but draws nothing into a page that has gone.
- **Old globals become aliases.** Each `window.*` name a page used to define
is made in `boot.js` with `alias(page, name, replacement)`, which forwards
to the module's export of the same name and warns once.
- **Timers are cleared in `destroy()`**, the one thing `ctx.signal` cannot
undo by itself.
`core/registry.js` handles the rest:
@@ -214,10 +235,10 @@ are the inline script in each partial today.
| # | Page | Inline JS | Why it is here |
|---|---|---|---|
| 1 | Cache (`cache.html`) | 163 lines, now 0 | **Done in stage 1.** One endpoint pair, no globals other pages use. The reference conversion |
| 2 | Rotation (`durations.html`) | 29 | Tiny. One htmx form |
| 3 | Operation History | 293 | No globals, read-only list |
| 4 | Config Editor (`raw_json.html`) | 212 | No globals. CodeMirror is set up and torn down in init/destroy |
| 5 | Backup & Restore | 232 | 5 globals used only by its own `onclick`s; these become delegated listeners plus deprecated aliases |
| 2 | Rotation (`durations.html`) | 29 lines, now 0 | **Done in stage 2.** The form stays plain htmx; the page starts the shared rotation-order widget, whose plugin-list request now takes `ctx.signal`. Its `hx-on` and `onsubmit` attributes call shared globals (`showSaveResult`, `fixInvalidNumberInputs`) and move with step 6 |
| 3 | Operation History | 293 lines, now 0 | **Done in stage 2.** Read-only list; rows drawn with `textContent`, the search debounce cleared on destroy. The "Showing x to y" counters now also reset when nothing matches |
| 4 | Config Editor (`raw_json.html`) | 212 lines, now 0 | **Done in stage 2.** Plain textareas (no CodeMirror on this page). It defined 5 globals after all (`formatJson`, `manualValidateJson`, `validateJSON`, `saveMainConfig`, `saveSecretsConfig`); nothing else used them, and they are deprecated aliases now. The live "Invalid JSON" line no longer puts the parser's message into `innerHTML` |
| 5 | Backup & Restore | 232 lines, now 0 | **Done in stage 2.** Its 5 globals (`exportBackup`, `loadBackupList`, `validateRestoreFile`, `clearRestore`, `runRestore`) are deprecated aliases; the buttons are delegated `data-action`s. Uploads go through `ctx.api.request(..., { body: formData })` (`api.js` gained a raw `body` option) |
| 6 | Schedule | 193 | 2 globals used as `hx-on` response handlers. Moves `hx-on` handlers into page listeners |
| 7 | General | 147 | `webLogin` global and the security section. The first page that touches login |
| 8 | Display | 231 | First page with `LEDVisibility` timers: those move to a `ctx.visibility` service that stops on destroy |
@@ -260,7 +281,11 @@ Unit suites need only node. They import the shipped modules directly:
| `unit/test_page_registry.js` | Unit, minimal DOM shim | The lifecycle: one init per root, destroy on swap, a veto keeps the page, swaps elsewhere leave it alone, the sweep, lazy loading, a destroy while loading, error containment |
| `unit/test_core_modules.js` | Unit | `api.js` (envelope, errors, abort, login redirect, path check) and `facade.js` (facade, aliases) |
| `dom/test_cache_page.js` | DOM: real partial, real API shape | No inline script; one request per swap and per Refresh after five swaps; a cancelled request draws nothing; hostile keys stay text; delete, empty, error, network and login states |
| `test/web_interface/test_es_modules.py` | pytest | MIME type; `no-cache` without `?v` and immutable with it; `boot.js` loads last; every import resolves inside `core/` and `pages/`; every registered page has its module and exactly one partial root; a converted partial has no `<script>` |
| `dom/test_durations_page.js` | DOM: real partial, real widget, real API shape | One plugin-list request per swap; Move down moves one place after five swaps; the swap cancels a request in flight; a late-loading widget is waited for, and a page swapped away while waiting starts nothing; hostile names stay text |
| `dom/test_operation_history_page.js` | DOM: real partial, real API shape | One history request per swap and per Refresh; the plugin filter filled once (from `PluginAPI`'s cache when loaded); paging, filters, debounced search, Clear (one DELETE), error/network/login states, cancel on swap; hostile ids, users and errors stay text |
| `dom/test_raw_json_page.js` | DOM: real partial, real config | One POST per Save after five swaps, to the right file; Format and Validate act once; invalid JSON never sent and its message stays text; a save survives a swap and is still reported; the old globals' entry points |
| `dom/test_backup_restore_page.js` | DOM: real partial, real API shape | One request per Refresh, Delete, Export (busy button ignores a second click), Inspect and Restore after five swaps; the upload's fields and the six restore options; reads cancelled by a swap, writes not; hostile file and host names stay text; the old globals' entry points |
| `test/web_interface/test_es_modules.py` | pytest | MIME type; `no-cache` without `?v` and immutable with it; `boot.js` loads last; every import resolves inside `core/` and `pages/`; the converted pages are exactly the registered ones, each with its module, `init`, and one root in the rendered partial; a converted partial has no `<script>` and no `onclick`; every moved global is aliased in `boot.js` and exported by its module, and no template defines it any more |
| `test/test_field_model_parity.py` | pytest | The model against the macro for every available schema |
What each future step adds:
+1
View File
@@ -47,6 +47,7 @@ src/config_service.py
src/core_config_keys.py
src/deprecation.py
src/device_location.py
src/display_arbiter.py
src/display_geometry.py
src/dynamic_team_resolver.py
src/exceptions.py
+4 -1
View File
@@ -175,7 +175,10 @@ class BaseOddsManager:
url = f"{self.base_url}/{sport}/leagues/{espn_league}/events/{event_id}/competitions/{event_id}/odds"
self.logger.debug(f"Requesting odds from URL: {url}")
response = fetch_get(self.session, url, timeout=self.request_timeout)
# The response cache may answer only inside this caller's own
# interval, the age at which its cached odds expire anyway.
response = fetch_get(self.session, url, timeout=self.request_timeout,
cache_max_age=interval)
response.raise_for_status()
raw_data = response.json()
+46 -13
View File
@@ -10,7 +10,12 @@ import logging
import time
from datetime import datetime
from types import MappingProxyType
from src.common.espn_dates import ESPN_MAX_LIMIT
from src.common.espn_dates import (
ESPN_MAX_LIMIT,
espn_scoreboard_cache_key,
read_espn_scoreboard_cache,
store_espn_scoreboard_cache,
)
from src.common.fetch_service import fetch_get, fetch_post, share_connection_pool
from typing import TYPE_CHECKING, Any, Dict, Mapping, Optional, cast
@@ -117,6 +122,14 @@ class APIHelper:
Returns:
Response data as dictionary or None if request fails
"""
return self._get(url, params, headers, timeout, cache_key, cache_ttl,
cache_ttl if cache_key else None)
def _get(self, url: str, params: Optional[Dict], headers: Optional[Dict],
timeout: Optional[int], cache_key: Optional[str], cache_ttl: int,
cache_max_age: Optional[float]) -> Optional[Dict]:
""":meth:`get`, saying how old a response the fetch service's short
response cache may hand back (``cache_max_age``, the caller's TTL)."""
if cache_key and self.cache_manager:
cached = self._get_from_cache(cache_key, cache_ttl)
if cached is not None:
@@ -138,7 +151,8 @@ class APIHelper:
url,
params=params,
headers=request_headers,
timeout=timeout or self.default_timeout
timeout=timeout or self.default_timeout,
cache_max_age=cache_max_age,
)
response.raise_for_status()
@@ -167,22 +181,23 @@ class APIHelper:
sport: Sport name (e.g., 'basketball', 'football')
league: League name (e.g., 'nba', 'nfl')
date: Date in YYYYMMDD format (defaults to today)
cache_key: Cache key for response
cache_ttl: Cache time-to-live in seconds
cache_key: Cache key for response. By default the canonical
``espn_scoreboard_cache_key(sport, league, date)``, shared
with every other consumer of this scoreboard, with the key
this used before (``espn_{sport}_{league}_{date}``) read as a
fallback for one release. An explicit key works as before.
cache_ttl: Cache time-to-live in seconds. A shared entry is
returned only while it is at most this old.
Returns:
ESPN API response data or None if request fails
"""
if date is None:
date = datetime.now().strftime('%Y%m%d')
# Build URL
url = f"https://site.api.espn.com/apis/site/v2/sports/{sport}/{league}/scoreboard"
# Build cache key if not provided
if cache_key is None:
cache_key = f"espn_{sport}_{league}_{date}"
# Set parameters
# limit above 500 makes ESPN truncate instead of erroring: college
# football came back with 25 of 68 games. See src/common/espn_dates.py.
@@ -190,8 +205,26 @@ class APIHelper:
'dates': date,
'limit': ESPN_MAX_LIMIT
}
return self.get(url, params=params, cache_key=cache_key, cache_ttl=cache_ttl)
if cache_key is not None:
return self.get(url, params=params, cache_key=cache_key, cache_ttl=cache_ttl)
legacy_key = f"espn_{sport}_{league}_{date}"
try:
shared_key = espn_scoreboard_cache_key(sport, league, date)
except ValueError:
# Not a path or date the canonical key covers: the old key.
return self.get(url, params=params, cache_key=legacy_key, cache_ttl=cache_ttl)
if self.cache_manager:
cached = read_espn_scoreboard_cache(
self.cache_manager, shared_key, cache_ttl, legacy_keys=(legacy_key,))
if cached is not None:
self.logger.debug(f"Using cached response for {shared_key}")
return cast(Dict[Any, Any], cached)
data = self._get(url, params, None, None, None, cache_ttl, cache_ttl)
if data is not None and self.cache_manager:
store_espn_scoreboard_cache(self.cache_manager, shared_key, data)
return data
def fetch_espn_standings(self, sport: str, league: str,
cache_key: Optional[str] = None,
+281 -10
View File
@@ -30,15 +30,34 @@ Once a range has been rejected, later ranges skip straight to chunks for
``RANGE_RETRY_SECONDS`` instead of spending a doomed request first -- live
scoreboards ask every 30 seconds. After that the range is tried again, so the
workaround retires itself if ESPN reverts.
ONE CACHE KEY PER SCOREBOARD
----------------------------
The same ESPN scoreboard used to be cached under a different key by every
consumer: odds-ticker as ``scoreboard_data_{sport}_{league}_{date}``,
``APIHelper`` as ``espn_{sport}_{league}_{date}``, the scoreboards as
``{sport_key}_schedule_{window}`` -- so two plugins showing the same league
fetched and stored it twice. :func:`espn_scoreboard_cache_key` is the one
name for "this sport/league scoreboard for these dates", and
:func:`get_espn_scoreboard` (or :func:`read_espn_scoreboard_cache` and
:func:`store_espn_scoreboard_cache` around :func:`fetch_espn_scoreboard`)
is the cache-through read every consumer can share. A read never returns an
entry older than the reader's own ``max_age``, whoever wrote it and whatever
ttl they stored with it. Old keys are passed as ``legacy_keys`` and read
after the canonical one, so an upgrade does not refetch everything at once;
they can go one release after the one that added this.
"""
import contextvars
import logging
import math
import re
import threading
import time
from concurrent.futures import ThreadPoolExecutor
from datetime import date, timedelta
from datetime import date, datetime, timedelta
from functools import partial
from typing import Any, Dict, List, Optional, Tuple, cast
from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, cast
try:
from src.common.json_body import response_json
@@ -51,18 +70,23 @@ except ImportError:
try:
# The core fetch service: counts, per-host budget, merging of identical
# requests. Same call, same result and errors as ``session.get``.
from src.common.fetch_service import fetch_get, pinned_caller
from src.common.fetch_service import fetch_get, get_fetch_service, pinned_caller
_COUNTS_FETCHES = True
except ImportError:
# Bundled copies on cores without it call the session directly.
import contextlib
def fetch_get(session: Any, url: str, *, share_in_flight: bool = True,
**kwargs: Any) -> Any:
cache_max_age: Optional[float] = None, **kwargs: Any) -> Any:
return session.get(url, **kwargs)
def pinned_caller() -> Any:
return contextlib.nullcontext()
_COUNTS_FETCHES = False
_logger = logging.getLogger(__name__)
# Above this, ESPN returns a truncated list instead of an error. See module
# docstring: 500 is the largest value measured to return complete data.
ESPN_MAX_LIMIT = 500
@@ -89,8 +113,23 @@ __all__ = [
"merge_scoreboard_payloads",
"fetch_espn_date_chunks",
"fetch_espn_scoreboard",
"ESPN_SCOREBOARD_URL",
"espn_scoreboard_url",
"espn_scoreboard_cache_key",
"espn_scoreboard_cache_key_for_url",
"read_espn_scoreboard_cache",
"store_espn_scoreboard_cache",
"get_espn_scoreboard",
]
#: The site-API scoreboard every sport and league shares.
ESPN_SCOREBOARD_URL = "https://site.api.espn.com/apis/site/v2/sports/{sport}/{league}/scoreboard"
_ESPN_HOST_URL = "https://site.api.espn.com/"
_PATH_PART = re.compile(r"^[a-z0-9][a-z0-9.\-]*$")
_DATES = re.compile(r"^\d{4}(?:\d{2}(?:\d{2})?)?$|^\d{8}-\d{8}$")
_SCOREBOARD_PATH = re.compile(r"/sports/([^/?#]+)/([^/?#]+)/scoreboard/?$")
def clamp_espn_limit(params: Optional[Dict[str, Any]]) -> Dict[str, Any]:
"""Return a copy of ``params`` with any ``limit`` over 500 pulled back to 500."""
@@ -129,6 +168,12 @@ def parse_espn_date_range(dates: Any) -> Optional[Tuple[date, date]]:
return start, end
def _memo_kwargs(cache_max_age: Optional[float]) -> Dict[str, Any]:
"""``cache_max_age`` for fetch_get, only when the caller gave one, so a
call that did not say is the call it always was."""
return {} if cache_max_age is None else {"cache_max_age": cache_max_age}
def _ranges_known_rejected() -> bool:
with _range_lock:
return time.monotonic() < _ranges_rejected_until
@@ -204,6 +249,7 @@ def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
def _fetch_one_chunk(
session, url: str, params: Dict[str, Any], headers, timeout, logger, chunk: str,
cache_max_age: Optional[float] = None,
) -> Optional[Dict[str, Any]]:
"""GET a single ``dates=`` chunk, or None when it failed.
@@ -217,6 +263,7 @@ def _fetch_one_chunk(
params=dict(params, dates=chunk, limit=ESPN_MAX_LIMIT),
headers=headers,
timeout=timeout,
**_memo_kwargs(cache_max_age),
)
response.raise_for_status()
return cast(Optional[Dict[str, Any]], response_json(response))
@@ -228,7 +275,7 @@ def _fetch_one_chunk(
def _fetch_chunks(
session, url: str, params: Dict[str, Any], headers, timeout, logger,
chunks: List[str],
chunks: List[str], cache_max_age: Optional[float] = None,
) -> List[Optional[Dict[str, Any]]]:
"""Fetch every chunk, returning payloads positionally aligned with ``chunks``.
@@ -246,6 +293,7 @@ def _fetch_chunks(
return []
fetch = partial(
_fetch_one_chunk, session, url, params, headers, timeout, logger,
cache_max_age=cache_max_age,
)
if len(chunks) == 1:
return [fetch(chunks[0])]
@@ -268,6 +316,7 @@ def fetch_espn_date_chunks(
headers: Optional[Dict[str, str]] = None,
timeout: int = 15,
logger=None,
cache_max_age: Optional[float] = None,
) -> Optional[Dict[str, Any]]:
"""Fetch a ``YYYYMMDD-YYYYMMDD`` window as month and day chunks.
@@ -299,7 +348,7 @@ def fetch_espn_date_chunks(
)
results = _fetch_chunks(
session, url, params, headers, timeout, logger, chunks,
session, url, params, headers, timeout, logger, chunks, cache_max_age,
)
attempted = len(chunks)
@@ -331,7 +380,7 @@ def fetch_espn_date_chunks(
days = [day for index in sorted(capped) for day in capped[index]]
attempted += len(days)
by_day = dict(zip(days, _fetch_chunks(
session, url, params, headers, timeout, logger, days,
session, url, params, headers, timeout, logger, days, cache_max_age,
)))
for index, month_days in capped.items():
slots[index] = [by_day.get(day) for day in month_days]
@@ -364,6 +413,7 @@ def fetch_espn_scoreboard(
headers: Optional[Dict[str, str]] = None,
timeout: int = 15,
logger=None,
cache_max_age: Optional[float] = None,
) -> Dict[str, Any]:
"""GET an ESPN scoreboard, re-asking in month/day chunks if a range 400s.
@@ -373,6 +423,10 @@ def fetch_espn_scoreboard(
and later ranges go straight to chunks for ``RANGE_RETRY_SECONDS``. A 400 on
a non-range request, any other error, and a range whose every chunk fails
all raise as before.
``cache_max_age`` is the oldest response, in seconds, the caller takes
from the fetch service's short response cache (its own TTL; 0 always
asks ESPN). None leaves it to the service default.
"""
params = clamp_espn_limit(params)
is_range = parse_espn_date_range(params.get("dates")) is not None
@@ -381,7 +435,7 @@ def fetch_espn_scoreboard(
if is_range and _ranges_known_rejected():
data = fetch_espn_date_chunks(
session, url, params=params, headers=headers,
timeout=timeout, logger=logger,
timeout=timeout, logger=logger, cache_max_age=cache_max_age,
)
if data is not None:
return data
@@ -389,7 +443,8 @@ def fetch_espn_scoreboard(
# real error to log, without spending the chunks a second time.
chunks_tried = True
response = fetch_get(session, url, params=params, headers=headers, timeout=timeout)
response = fetch_get(session, url, params=params, headers=headers, timeout=timeout,
**_memo_kwargs(cache_max_age))
if is_range and response.status_code == 400 and not chunks_tried:
_note_range_rejected()
if logger:
@@ -400,9 +455,225 @@ def fetch_espn_scoreboard(
)
data = fetch_espn_date_chunks(
session, url, params=params, headers=headers,
timeout=timeout, logger=logger,
timeout=timeout, logger=logger, cache_max_age=cache_max_age,
)
if data is not None:
return data
response.raise_for_status()
return cast(Dict[str, Any], response_json(response))
# --- one cache key per scoreboard --------------------------------------------------
def espn_scoreboard_url(sport: str, league: str) -> str:
"""The site-API scoreboard URL for an ESPN ``sport`` / ``league`` path."""
return ESPN_SCOREBOARD_URL.format(sport=_path_part(sport, "sport"),
league=_path_part(league, "league"))
def _path_part(value: Any, what: str) -> str:
text = str(value or "").strip().lower()
if not _PATH_PART.match(text):
raise ValueError(f"not an ESPN {what} path segment: {value!r}")
return text
def _day(value: Any) -> str:
if isinstance(value, (date, datetime)):
return value.strftime("%Y%m%d")
text = str(value).strip()
if len(text) != 8 or not text.isdigit():
raise ValueError(f"not an ESPN day (YYYYMMDD): {value!r}")
return text
def _dates_part(dates: Any) -> str:
"""``dates`` as ESPN spells it, or ``current`` for no ``dates`` at all."""
if dates is None or dates == "":
return "current"
if isinstance(dates, (date, datetime)):
return _day(dates)
if isinstance(dates, (tuple, list)):
if len(dates) != 2:
raise ValueError(f"a date range is (start, end): {dates!r}")
start, end = _day(dates[0]), _day(dates[1])
return start if start == end else f"{start}-{end}"
text = str(dates).strip()
if isinstance(dates, bool) or not _DATES.match(text):
raise ValueError(
f"not an ESPN dates value (YYYY, YYYYMM, YYYYMMDD or "
f"YYYYMMDD-YYYYMMDD): {dates!r}")
return text
def espn_scoreboard_cache_key(sport: str, league: str, dates: Any = None) -> str:
"""The one cache key for an ESPN scoreboard, whoever caches it.
``sport`` and ``league`` are ESPN's own path segments -- ``football`` /
``college-football``, ``soccer`` / ``eng.1`` -- not a plugin's
``sport_key``, so every plugin showing a league names it the same way.
``dates`` is what the request sends as ``dates=``: ``"YYYYMMDD"``,
``"YYYYMM"``, ``"YYYY"``, ``"YYYYMMDD-YYYYMMDD"``, a ``date``, or a
``(start, end)`` pair of either; None is the undated "current"
scoreboard. Anything else raises ValueError rather than invent a key.
The key says nothing about ``limit``: a cached copy is meant to be a
whole one (the helpers here always ask for ``ESPN_MAX_LIMIT``).
"""
return (f"espn_scoreboard_{_path_part(sport, 'sport')}_"
f"{_path_part(league, 'league')}_{_dates_part(dates)}")
def espn_scoreboard_cache_key_for_url(url: str, dates: Any = None) -> Optional[str]:
""":func:`espn_scoreboard_cache_key` for a scoreboard URL, or None when
``url`` is not ``.../sports/{sport}/{league}/scoreboard``."""
match = _SCOREBOARD_PATH.search(str(url or "").split("?", 1)[0])
if match is None:
return None
try:
return espn_scoreboard_cache_key(match.group(1), match.group(2), dates)
except ValueError:
return None
def _note_cache_hit(legacy: bool, avoided_request: bool = True) -> None:
if not _COUNTS_FETCHES:
return
try:
get_fetch_service().note_cache_hit(
_ESPN_HOST_URL, legacy=legacy, avoided_request=avoided_request)
except Exception: # noqa: BLE001 - counting never breaks a read
_logger.debug("could not count a scoreboard cache hit", exc_info=True)
def _fresh_cached(cache_manager: Any, key: str, max_age: Optional[float],
now: float) -> Tuple[Optional[Dict[str, Any]], Optional[float]]:
"""The data cached under ``key`` if it is at most ``max_age`` seconds
old, and its age (None when the cache does not say).
The age is the stored record's own timestamp, checked here: CacheManager
lets a ttl stored by the writer override the reader's max_age, and its
memory tier times an entry from when it was loaded, not written. A key
shared by readers with different TTLs can rely on neither.
"""
reader = getattr(cache_manager, "get_cached_data", None)
limit = None if max_age is None else max(1, int(math.ceil(max_age)))
if not callable(reader):
# A cache without records (a test double, a plugin's own store).
value = cache_manager.get(key, max_age=limit)
return (value if isinstance(value, dict) else None), None
record = reader(key, max_age=limit, memory_ttl=limit)
if not isinstance(record, dict):
return None, None
if "data" not in record:
return record, None # unwrapped; the cache already judged it by mtime
stamp = record.get("timestamp")
age: Optional[float] = None
if not isinstance(stamp, bool) and isinstance(stamp, (int, float)):
age = max(0.0, now - float(stamp))
if max_age is not None and (age is None or age > max_age):
return None, None
data = record["data"]
return (data if isinstance(data, dict) else None), age
def read_espn_scoreboard_cache(
cache_manager: Any,
key: str,
max_age: Optional[float],
legacy_keys: Iterable[str] = (),
now: Optional[float] = None,
accept: Optional[Callable[[Dict[str, Any], Optional[float]], bool]] = None,
) -> Optional[Dict[str, Any]]:
"""The cached scoreboard under ``key``, or under the first of
``legacy_keys`` that has one, if it is at most ``max_age`` seconds old.
None on a miss, a stale entry, ``max_age`` of 0 or less, no cache
manager, or any cache error -- a read never raises. ``max_age=None``
takes an entry of any age. ``accept(data, age_seconds)`` can turn down
an entry the age alone would allow (a payload holding a live game wants
a shorter limit); ``age_seconds`` is None when the cache cannot say. A
hit is counted in the fetch statistics (``cache_hits``;
``legacy_cache_hits`` too for an old key).
"""
if cache_manager is None:
return None
if max_age is not None and max_age <= 0:
return None
clock = time.time() if now is None else now
for index, candidate in enumerate([key, *legacy_keys]):
if not candidate:
continue
try:
data, age = _fresh_cached(cache_manager, candidate, max_age, clock)
if data is not None and accept is not None and not accept(data, age):
data = None
except Exception: # noqa: BLE001 - a broken cache is a miss
_logger.debug("scoreboard cache read failed for %s", candidate, exc_info=True)
continue
if data is not None:
_note_cache_hit(legacy=index > 0)
return data
return None
def store_espn_scoreboard_cache(cache_manager: Any, key: str, data: Any) -> None:
"""Cache a fetched scoreboard under ``key``. Never raises.
No ttl is stored: each reader applies its own ``max_age`` (a live
reader 30 s, a schedule reader an hour), and a stored ttl would
override theirs in CacheManager.
"""
if cache_manager is None or data is None:
return
try:
cache_manager.set(key, data)
except Exception: # noqa: BLE001 - the caller still has its data
_logger.warning("Could not cache scoreboard %s", key, exc_info=True)
def get_espn_scoreboard(
session: Any,
sport: str,
league: str,
dates: Any = None,
*,
cache_manager: Any = None,
max_age: Optional[float] = 300,
legacy_keys: Iterable[str] = (),
headers: Optional[Dict[str, str]] = None,
timeout: int = 15,
logger: Any = None,
) -> Dict[str, Any]:
"""An ESPN scoreboard through the shared cache, fetched on a miss.
Reads :func:`espn_scoreboard_cache_key` (then ``legacy_keys``) and
returns an entry at most ``max_age`` seconds old. Otherwise it fetches
with :func:`fetch_espn_scoreboard` -- ``limit=ESPN_MAX_LIMIT``, ranges
split as ESPN needs -- caches the result under the canonical key and
returns it. ``max_age=0`` always fetches (and still caches, for other
readers). Errors raise exactly as :func:`fetch_espn_scoreboard` does,
and nothing is cached then. ``session=None`` uses the fetch service's
pooled session for the ESPN host.
"""
key = espn_scoreboard_cache_key(sport, league, dates)
cached = read_espn_scoreboard_cache(cache_manager, key, max_age, legacy_keys)
if cached is not None:
return cast(Dict[str, Any], cached)
params: Dict[str, Any] = {"limit": ESPN_MAX_LIMIT}
spelled = _dates_part(dates)
if spelled != "current":
params["dates"] = spelled
data = fetch_espn_scoreboard(
session,
espn_scoreboard_url(sport, league),
params=params,
headers=headers,
timeout=timeout,
logger=logger,
# The response cache must not hand back anything older than the
# cache read above would have accepted.
cache_max_age=None if max_age is None else max(0.0, float(max_age)),
)
store_espn_scoreboard_cache(cache_manager, key, data)
return data
+238 -12
View File
@@ -46,9 +46,25 @@ own conditional headers gets the raw answer. (ESPN sent no validators when
this was written -- see the PR that added this module -- so on ESPN the store
stays empty and costs nothing.)
**Response cache (stage 2).** A 200 that says ``Cache-Control: max-age=N``
is kept in memory for those N seconds (less its ``Age``), and an identical
GET inside that window is answered from it without a request. ESPN sends
max-age (1-496 s measured on 2026-10-02, most of it under 10 s) and no
validators, so this is the only revalidation-free reuse ESPN allows. It
never hands a caller a response older than the caller accepts: a caller
says how old with ``cache_max_age`` (``fetch_get(..., cache_max_age=ttl)``;
0 skips the cache), and one that does not say gets at most
``response_cache.default_max_age`` (30 s). ``no-store``, ``no-cache``,
``private``, ``Vary: *`` and ``Set-Cookie`` responses are never kept.
Identical means what the validator store keys on: URL, query, effective
headers and, for a session with cookies or auth, the session.
**Counters.** Requests, merged requests, bytes, 304s, errors, HTTP errors,
adapter retries, throttled requests and seconds waited, per plugin and per
host. :class:`FetchStatsPublisher` publishes them for the web interface
adapter retries, throttled requests and seconds waited, plus requests
answered without the network: ``memo_hits`` (the response cache) and
``cache_hits`` / ``legacy_cache_hits`` (a shared ESPN scoreboard cache entry,
counted by ``src/common/espn_dates.py``). Per plugin and per host.
:class:`FetchStatsPublisher` publishes them for the web interface
(``GET /api/v3/plugins/fetch-stats``).
CALLER IDENTITY
@@ -68,7 +84,8 @@ Counters are kept per plugin without plugins saying who they are:
3. Otherwise the request is the core's own (``"core"``).
Nothing here raises on account of bookkeeping: a failure in counting,
keying or the validator store falls back to a plain ``session.get``.
keying, the validator store or the response cache falls back to a plain
``session.get``.
Core-internal for now (stage 1). Plugins reach it through ``APIHelper`` and
``espn_dates``; a plugin-facing API comes with stage 3.
@@ -143,8 +160,22 @@ DEFAULT_CONFIG: Mapping[str, Any] = {
"max_bytes": 4 * 1024 * 1024,
"max_entry_bytes": 1024 * 1024,
},
# Stage 2: responses ESPN calls fresh (Cache-Control: max-age), reused
# for identical GETs. A college-football Saturday is ~1 MB decoded, so
# one entry may be 2 MB; months (5-7 MB) are never kept.
"response_cache": {
"enabled": True,
"default_max_age": 30,
"max_entries": 64,
"max_bytes": 6 * 1024 * 1024,
"max_entry_bytes": 2 * 1024 * 1024,
},
}
#: However long a server says a response stays fresh, it is not kept longer
#: than this: the cache is for requests that coincide, not for storage.
_RESPONSE_CACHE_CEILING = 600.0
#: Connection pools kept per shared adapter (one per host) and connections
#: kept per pool. Larger than requests' 10 because one adapter now serves
#: every core Session with its retry policy: three background workers each
@@ -171,6 +202,9 @@ _COUNTER_FIELDS = (
"overruns", # requests that went after max_wait_seconds anyway
"bytes", # decoded response body bytes received
"wait_seconds", # time spent waiting for host budgets
"memo_hits", # answered from the response cache (max-age); nothing sent
"cache_hits", # scoreboard fetches answered from a shared ESPN cache entry
"legacy_cache_hits", # cache reads answered from a pre-stage-2 key (any helper)
)
@@ -393,6 +427,117 @@ class _ValidatorStore:
return {"entries": len(self._entries), "bytes": self._bytes}
# --- response cache (Cache-Control: max-age) --------------------------------------
@dataclass
class _Fresh:
response: requests.Response
stored_at: float
#: Seconds after stored_at the server said the response stays fresh.
lifetime: float
size: int
class _ResponseCache:
"""LRU of finished 200 responses, each kept for its server max-age.
:meth:`get` answers only while the entry is younger than both its own
lifetime and the caller's limit, so nobody is handed a response older
than they asked for. Expired entries are dropped as they are met and on
every insert, so the cache holds only what is still fresh.
"""
def __init__(self, max_entries: int, max_bytes: int, max_entry_bytes: int,
clock: Callable[[], float]) -> None:
self.max_entries = max_entries
self.max_bytes = max_bytes
self.max_entry_bytes = max_entry_bytes
self._clock = clock
self._entries: "OrderedDict[Any, _Fresh]" = OrderedDict()
self._bytes = 0
self._lock = threading.Lock()
def get(self, key: Any, max_age: float) -> Optional[requests.Response]:
with self._lock:
entry = self._entries.get(key)
if entry is None:
return None
age = self._clock() - entry.stored_at
if age < 0 or age >= entry.lifetime:
self._drop_locked(key)
return None
if age > max_age:
return None # fresh for someone less strict; kept
self._entries.move_to_end(key)
clone: requests.Response = _clone_response(entry.response)
return clone
def put(self, key: Any, response: requests.Response, lifetime: float,
size: int) -> None:
with self._lock:
self._drop_locked(key)
if size > self.max_entry_bytes or self.max_entries <= 0 or lifetime <= 0:
return
now = self._clock()
for old_key in [k for k, e in self._entries.items()
if now - e.stored_at >= e.lifetime]:
self._drop_locked(old_key)
self._entries[key] = _Fresh(_clone_response(response), now, lifetime, size)
self._bytes += size
while self._entries and (len(self._entries) > self.max_entries
or self._bytes > self.max_bytes):
_, old = self._entries.popitem(last=False)
self._bytes -= old.size
def _drop_locked(self, key: Any) -> None:
old = self._entries.pop(key, None)
if old is not None:
self._bytes -= old.size
def clear(self) -> None:
with self._lock:
self._entries.clear()
self._bytes = 0
def stats(self) -> Dict[str, int]:
with self._lock:
return {"entries": len(self._entries), "bytes": self._bytes}
def _cache_directives(value: Optional[str]) -> Dict[str, Optional[str]]:
directives: Dict[str, Optional[str]] = {}
for part in (value or "").split(","):
name, _, arg = part.strip().partition("=")
if name:
directives[name.strip().lower()] = arg.strip().strip('"') if arg else None
return directives
def _fresh_for(response: Any) -> Optional[float]:
"""Seconds a finished response stays fresh by its own headers, or None
when it must not be reused: not a 200 with its body read, ``no-store``,
``no-cache``, ``private``, ``Vary: *``, ``Set-Cookie``, or no max-age."""
if _status_of(response) != 200 or _body_of(response) is None:
return None
directives = _cache_directives(_str_header(response, "Cache-Control"))
if {"no-store", "no-cache", "private"} & set(directives):
return None
if (_str_header(response, "Vary") or "").strip() == "*":
return None
if _str_header(response, "Set-Cookie"):
return None
try:
max_age = int(directives.get("max-age") or "")
except ValueError:
return None
try:
age = int(_str_header(response, "Age") or 0)
except ValueError:
age = 0
fresh = float(min(max_age - max(age, 0), _RESPONSE_CACHE_CEILING))
return fresh if fresh > 0 else None
# --- helpers ------------------------------------------------------------------------
def _host_of(url: Any) -> str:
@@ -580,6 +725,9 @@ class FetchService:
self.max_wait_seconds = 2.0
self._rate_limits: Dict[str, Tuple[float, float]] = {}
self._validators = _ValidatorStore(0, 0, 0)
self.response_cache = True
self.default_max_age = 30.0
self._fresh = _ResponseCache(0, 0, 0, clock)
self._applied: Optional[str] = None
self.configure(config)
@@ -635,6 +783,14 @@ class FetchService:
store = merged.get("validator_store")
store = store if isinstance(store, Mapping) else {}
default_store = DEFAULT_CONFIG["validator_store"]
fresh = merged.get("response_cache")
if fresh is not None and not isinstance(fresh, Mapping):
logger.warning("fetch_service.response_cache is not an object; using the defaults")
fresh = fresh if isinstance(fresh, Mapping) else {}
default_fresh = DEFAULT_CONFIG["response_cache"]
self.response_cache = fresh.get("enabled") is not False
self.default_max_age = _as_float(fresh.get("default_max_age"),
float(default_fresh["default_max_age"]))
with self._lock:
self._rate_limits = limits
self._buckets.clear()
@@ -643,6 +799,12 @@ class FetchService:
_as_int(store.get("max_bytes"), default_store["max_bytes"]),
_as_int(store.get("max_entry_bytes"), default_store["max_entry_bytes"]),
)
self._fresh = _ResponseCache(
_as_int(fresh.get("max_entries"), default_fresh["max_entries"]),
_as_int(fresh.get("max_bytes"), default_fresh["max_bytes"]),
_as_int(fresh.get("max_entry_bytes"), default_fresh["max_entry_bytes"]),
self._clock,
)
self.change_count += 1
def describe_config(self) -> Dict[str, Any]:
@@ -655,6 +817,8 @@ class FetchService:
"conditional_get": self.conditional_get,
"max_wait_seconds": self.max_wait_seconds,
"rate_limits": limits,
"response_cache": self.response_cache,
"default_max_age": self.default_max_age,
}
def _limit_for(self, host: str) -> Optional[Tuple[float, float]]:
@@ -723,7 +887,7 @@ class FetchService:
# -- requests --
def get(self, session: Any, url: str, *, share_in_flight: bool = True,
**kwargs: Any) -> Any:
cache_max_age: Optional[float] = None, **kwargs: Any) -> Any:
"""``session.get(url, **kwargs)`` through the service.
Same return value, same exceptions, and ``session.get`` is called
@@ -734,6 +898,11 @@ class FetchService:
than joining an identical one in flight -- for a caller that may
retry *because* an earlier request hung (BackgroundDataService
cancels and replaces a fetch) and must not be handed that one.
``cache_max_age`` is the oldest response, in seconds, the caller
will take from the response cache (its own TTL); 0 always asks the
network. None means ``response_cache.default_max_age``. A response
is never reused past the max-age its server gave it either.
"""
transport = session if session is not None else self.session_for(url)
if not self.enabled:
@@ -744,8 +913,24 @@ class FetchService:
logger.debug("fetch_service could not key a request to %s", url, exc_info=True)
request = _Request(plugin=self._caller(), host=_host_of(url))
reusable = (self.response_cache and request.representation is not None
and not _has_conditional_headers(kwargs.get("headers")))
if reusable:
try:
limit = self._accepted_age(cache_max_age)
cached = self._fresh.get(request.representation, limit) if limit > 0 else None
except Exception:
logger.debug("fetch_service response cache lookup failed", exc_info=True)
cached = None
if cached is not None:
self._count(request.plugin, request.host, memo_hits=1)
return cached
if request.flight is None or not self.single_flight or not share_in_flight:
return self._send_get(transport, url, kwargs, request)
response = self._send_get(transport, url, kwargs, request)
if reusable:
self._remember(request, response)
return response
with self._lock:
flight = self._inflight.get(request.flight)
@@ -764,6 +949,8 @@ class FetchService:
try:
response = self._send_get(transport, url, kwargs, request)
flight.response = response
if reusable:
self._remember(request, response)
return response
except BaseException as error:
flight.error = error
@@ -790,6 +977,40 @@ class FetchService:
except Exception:
logger.debug("fetch_service could not count a merged request", exc_info=True)
def note_cache_hit(self, url: Any = None, *, legacy: bool = False,
avoided_request: bool = True,
plugin_id: Optional[str] = None) -> None:
"""Count a read answered from a shared cache entry instead of the
network (``espn_dates``). ``legacy`` marks a read from a key that
predates the canonical one; ``avoided_request=False`` counts only
that, for a read that was never going to fetch on a miss."""
try:
self._count(plugin_id or self._caller(), _host_of(url),
cache_hits=int(avoided_request), legacy_cache_hits=int(legacy))
except Exception:
logger.debug("fetch_service could not count a cache hit", exc_info=True)
def _accepted_age(self, cache_max_age: Any) -> float:
"""The oldest cached response this call accepts, in seconds."""
if cache_max_age is None:
return self.default_max_age
if isinstance(cache_max_age, bool) or not isinstance(cache_max_age, (int, float)):
return self.default_max_age
value = float(cache_max_age)
return value if math.isfinite(value) and value > 0 else 0.0
def _remember(self, request: _Request, response: Any) -> None:
"""Keep a finished response for its server max-age. Never raises."""
try:
lifetime = _fresh_for(response)
if lifetime is None:
return
body = _body_of(response)
self._fresh.put(request.representation, response, lifetime,
len(body) if body is not None else 0)
except Exception:
logger.debug("fetch_service could not keep a response", exc_info=True)
def _caller(self) -> str:
return current_plugin_id() or CORE
@@ -927,10 +1148,11 @@ class FetchService:
per_plugin[name] += value
per_host[name] += value
self._totals[name] += value
if changes.get("requests") or changes.get("merged"):
asked = int(changes.get("requests", 0) + changes.get("merged", 0)
+ changes.get("memo_hits", 0) + changes.get("cache_hits", 0))
if asked:
hosts = self._plugin_hosts.setdefault(plugin, {})
hosts[host] = hosts.get(host, 0) + int(changes.get("requests", 0)
+ changes.get("merged", 0))
hosts[host] = hosts.get(host, 0) + asked
self.change_count += 1
def reset_counters(self) -> None:
@@ -942,12 +1164,14 @@ class FetchService:
self.change_count += 1
def reset(self) -> None:
"""Counters, validators, budgets and in-flight table (tests)."""
"""Counters, validators, response cache, budgets and in-flight
table (tests)."""
self.reset_counters()
with self._lock:
self._buckets.clear()
self._inflight.clear()
self._validators.clear()
self._fresh.clear()
def snapshot(self) -> Dict[str, Any]:
"""Counters since the service started, JSON-ready."""
@@ -971,6 +1195,7 @@ class FetchService:
"plugins": plugins,
"hosts": hosts,
"validators": self._validators.stats(),
"response_cache": self._fresh.stats(),
"config": self.describe_config(),
}
@@ -1005,10 +1230,11 @@ def configure_fetch_service(config: Any) -> FetchService:
def fetch_get(session: Any, url: str, *, share_in_flight: bool = True,
**kwargs: Any) -> Any:
"""``session.get(url, **kwargs)`` through the process's FetchService."""
cache_max_age: Optional[float] = None, **kwargs: Any) -> Any:
"""``session.get(url, **kwargs)`` through the process's FetchService.
``cache_max_age``: see :meth:`FetchService.get`."""
return get_fetch_service().get(session, url, share_in_flight=share_in_flight,
**kwargs)
cache_max_age=cache_max_age, **kwargs)
def fetch_post(session: Any, url: str, **kwargs: Any) -> Any:
+89 -3
View File
@@ -40,6 +40,20 @@ listed here.
- ``live_games``, read with ``getattr`` -- ``_needs_previous_day``.
- ``background_service``, read with ``getattr`` --
``_background_fetches_espn_ranges``.
- ``sport`` and ``league`` (ESPN's path segments, e.g. ``football`` /
``nfl``) -- ``_schedule_cache_key``, and ``_fetch_season_directly`` when
it is given no key and cannot read one from its URL.
THE SCHEDULE CACHE KEY (fetch service stage 2)
----------------------------------------------
``_schedule_cache_key`` names a schedule window with the canonical
``espn_scoreboard_cache_key`` instead of a plugin-built
``{sport_key}_schedule_{window}``, and ``_cached_schedule`` reads it with the
old key as a fallback for one release, so an upgrade serves the copy already
on disk instead of refetching every league at once. The canonical key
carries the window's dates, so it moves on a day as the window slides; a
miss on it also deletes the copy for the day before, so a league keeps one
window file instead of a week of them.
Add it as a base of the plugin's ``SportsCore``, e.g.
``class SportsCore(SportsFetchMixin, SportsCoreSharedMixin,
@@ -50,9 +64,31 @@ constant on the plugin's own class still wins over the mixin's.
import logging
import threading
from datetime import datetime, timedelta
from typing import Any, ClassVar, Dict, Optional
from typing import Any, ClassVar, Dict, Iterable, Optional
from src.common.espn_dates import ESPN_MAX_LIMIT, fetch_espn_scoreboard
from src.common.espn_dates import (
ESPN_MAX_LIMIT,
espn_scoreboard_cache_key,
espn_scoreboard_cache_key_for_url,
fetch_espn_scoreboard,
parse_espn_date_range,
)
from src.common.fetch_service import get_fetch_service
_ESPN_SITE = "https://site.api.espn.com/"
def _previous_window_key(cache_key: str) -> Optional[str]:
"""The canonical key of the same window one day earlier, or None when
``cache_key`` is not a canonical day-range key."""
head, sep, dates = cache_key.rpartition("_")
if not sep or not head.startswith("espn_scoreboard_"):
return None
span = parse_espn_date_range(dates)
if span is None:
return None
start, end = (day - timedelta(days=1) for day in span)
return f"{head}_{start.strftime('%Y%m%d')}-{end.strftime('%Y%m%d')}"
class SportsFetchMixin:
@@ -65,6 +101,8 @@ class SportsFetchMixin:
cache_manager: Any
logger: logging.Logger
_games_lock: threading.RLock
sport: str
league: str
#: How many games past the one on screen keep their odds warm. One is
#: enough for the line to be ready when the rotation advances; more just
@@ -154,18 +192,66 @@ class SportsFetchMixin:
service = getattr(self, "background_service", None)
return bool(getattr(service, "handles_espn_date_ranges", False))
def _schedule_cache_key(self, datestring: str) -> str:
"""The canonical cache key for this league's schedule over
``datestring`` (``espn_scoreboard_cache_key``)."""
return espn_scoreboard_cache_key(self.sport, self.league, datestring)
def _cached_schedule(self, cache_key: str, legacy_keys: Iterable[str] = ()) -> Any:
"""What ``self.cache_manager.get(cache_key)`` returns, falling back
to each of ``legacy_keys`` (the plugin's pre-canonical keys) in turn.
The same read the managers made before -- same default max age, a
stored ttl still wins -- so moving to the canonical key changes
where a schedule is cached, not for how long. A read from an old key
is counted (``legacy_cache_hits``) so it is visible when the
fallback can go. A miss on the canonical key also deletes the same
window's copy from the day before (see the module docstring).
"""
cached = self.cache_manager.get(cache_key)
if cached:
return cached
self._retire_previous_window(cache_key)
for legacy in legacy_keys:
if not legacy or legacy == cache_key:
continue
cached = self.cache_manager.get(legacy)
if cached:
try:
get_fetch_service().note_cache_hit(
_ESPN_SITE, legacy=True, avoided_request=False)
except Exception: # noqa: BLE001 - counting never breaks a read
pass
return cached
return None
def _retire_previous_window(self, cache_key: str) -> None:
previous = _previous_window_key(cache_key)
delete = getattr(self.cache_manager, "delete", None)
if previous is None or not callable(delete):
return
try:
delete(previous)
except Exception as e: # noqa: BLE001 - housekeeping only
self.logger.debug(f"Could not delete old schedule copy {previous}: {e}")
def _fetch_season_directly(
self,
url: str,
datestring: str,
cache_key: str,
cache_key: Optional[str],
label: str,
ttl: Optional[int] = None,
) -> Optional[Dict]:
"""Fetch a season schedule on this thread, in chunks ESPN accepts, and cache it.
``label`` names the schedule in log lines, e.g. ``"2026 season"``.
``cache_key=None`` caches it under the canonical key
(``espn_scoreboard_cache_key`` for ``url``'s sport and league).
"""
if cache_key is None:
cache_key = (espn_scoreboard_cache_key_for_url(url, datestring)
or self._schedule_cache_key(datestring))
try:
data = fetch_espn_scoreboard(
self.session,
+185
View File
@@ -0,0 +1,185 @@
"""What the panel shows next: the Arbiter of docs/RUN_LOOP_REDESIGN.md.
``Arbiter.decide(state, inputs, now)`` takes a snapshot that
``DisplayController.run()`` gathers once per pass and returns a
:class:`ScreenPlan` naming the Source that gets the panel. It is a pure
function: no I/O, no clock reads (``now`` is passed in), no locks, and it
changes nothing it is given. That is what lets a plain table of cases test
the priority order, which used to exist only as the order of ``if`` blocks
in ``run()``.
The full order is
ScheduledOff (a gate), Follower, OnDemand, Wifi, Live, Vegas, Rotation
Stage 2 decides the gate, Follower and Wifi. Every other case returns a
``LEGACY`` plan, meaning "carry on with run()'s existing code" (live
priority, Vegas, then one rotation screen). OnDemand is in the order already
because it outranks the WiFi notice: an active session is a ``LEGACY`` plan
even when a notice is pending.
The Wifi Source's mid-screen rule, :func:`wifi_notice_preempts`, lives here
too, so both of its answers -- at the top of a pass and between frames --
come from one module.
"""
from dataclasses import dataclass
from enum import Enum
from typing import Optional
__all__ = [
"Arbiter",
"ArbiterInputs",
"ArbiterState",
"SCHEDULED_OFF_DWELL",
"ScreenPlan",
"Source",
"WIFI_NOTICE_DWELL",
"WifiNotice",
"wifi_notice_preempts",
]
# How long one scheduled-off pass blanks the panel. The dwell ends early when
# on-demand starts or the schedule turns the panel back on.
SCHEDULED_OFF_DWELL = 60.0
# How long one WiFi-notice pass holds the notice before the next pass looks
# again; the notice stays up, pass after pass, until it expires.
WIFI_NOTICE_DWELL = 0.5
class Source(Enum):
"""Who gets the panel this pass."""
SCHEDULED_OFF = "scheduled-off"
FOLLOWER = "follower"
WIFI = "wifi"
# Not decided by the Arbiter yet: on-demand, live priority, Vegas and the
# rotation are still chosen by run()'s own code. Stage 3 adds the
# OnDemand, Live and Rotation Sources; stage 4 adds Vegas.
LEGACY = "legacy"
@dataclass(frozen=True)
class WifiNotice:
"""A WiFi status message waiting to be drawn.
``expires_at`` is wall-clock time (``time.time()``), as written by the
WiFi manager.
"""
message: str
expires_at: float
@dataclass(frozen=True)
class ArbiterState:
"""What the Arbiter remembers between passes.
Nothing yet: the stage-2 Sources decide from the inputs alone. The
on-demand index, the rotation index and the live resume point move here
with their Sources in stage 3.
"""
@dataclass(frozen=True)
class ArbiterInputs:
"""One pass's snapshot, gathered by run() before it calls decide().
Attributes:
schedule_on: The display schedule has the panel on, not counting an
on-demand override of a scheduled-off window.
on_demand_active: An on-demand session is running.
follower_active: A sync leader is driving this panel.
wifi_notice: The pending WiFi notice, or None. run() reads it only
when it could win (the panel is on, and neither a follower nor
on-demand outranks it), because reading it has side effects: a
1 Hz throttle and deleting an expired file.
"""
schedule_on: bool
on_demand_active: bool
follower_active: bool
wifi_notice: Optional[WifiNotice] = None
@dataclass(frozen=True)
class ScreenPlan:
"""The Arbiter's answer for one pass.
Attributes:
source: The Source that gets the panel.
max_duration: How long the plan holds the panel, in seconds, at most
(its dwell ends early when what the panel should show changes).
None when the Source paces itself: a follower frame, or LEGACY.
notice: The WiFi notice to draw, for a WIFI plan.
"""
source: Source
max_duration: Optional[float] = None
notice: Optional[WifiNotice] = None
SCHEDULED_OFF_PLAN = ScreenPlan(Source.SCHEDULED_OFF, max_duration=SCHEDULED_OFF_DWELL)
FOLLOWER_PLAN = ScreenPlan(Source.FOLLOWER)
LEGACY_PLAN = ScreenPlan(Source.LEGACY)
class Arbiter:
"""Decides which Source gets the panel. Stateless; see the module docstring."""
@staticmethod
def decide(state: ArbiterState, inputs: ArbiterInputs, now: float) -> ScreenPlan:
"""The plan for this pass, from the Sources in priority order.
Args:
state: What the Arbiter remembers between passes (nothing yet).
inputs: This pass's snapshot.
now: Wall-clock time of the snapshot. No stage-2 Source reads it:
the top-of-pass WiFi check takes the notice as read, and only
the mid-screen check (:func:`wifi_notice_preempts`) compares
it with the expiry. It is in the signature for the Sources
stage 3 adds (on-demand expiry, durations).
Returns:
The winning Source's plan, or LEGACY_PLAN when the winner is one
run() still decides itself.
"""
del state, now # not read by the stage-2 Sources; see the docstring
# ScheduledOff is a gate, not a Source: a scheduled-off panel stays
# blank even for a follower, and only an on-demand session overrides
# it (#714 -- one ending in off hours blanks at the next pass).
if not inputs.schedule_on and not inputs.on_demand_active:
return SCHEDULED_OFF_PLAN
# 1. Follower: a sync leader drives this panel, ahead of on-demand.
if inputs.follower_active:
return FOLLOWER_PLAN
# 2. OnDemand: decided by run() until stage 3. It outranks the notice.
if inputs.on_demand_active:
return LEGACY_PLAN
# 3. Wifi: a pending notice, held for one short dwell per pass.
if inputs.wifi_notice is not None:
return ScreenPlan(Source.WIFI, max_duration=WIFI_NOTICE_DWELL,
notice=inputs.wifi_notice)
# 4-6. Live, Vegas, Rotation: still run()'s own code.
return LEGACY_PLAN
def wifi_notice_preempts(notice: Optional[WifiNotice], on_demand_active: bool,
now: float) -> bool:
"""Whether a WiFi notice should end the current screen early.
The Wifi Source's mid-screen rule, polled between frames, during dwells
and when a Vegas iteration yields. On-demand outranks the notice, as in
:meth:`Arbiter.decide`. Unlike the top-of-pass check it also compares
``now`` with the expiry, because the 1 Hz read throttle can hand back a
notice that has expired since it was read.
"""
if on_demand_active or notice is None:
return False
return now < notice.expires_at
+78 -30
View File
@@ -35,6 +35,9 @@ from concurrent.futures import ThreadPoolExecutor, as_completed # pylint: disab
import pytz
from src import display_watchdog
from src.display_arbiter import (
Arbiter, ArbiterInputs, ArbiterState, Source, WifiNotice, wifi_notice_preempts,
)
from src.display_manager import DisplayManager
from src.config_manager import ConfigManager
from src.config_service import ConfigService
@@ -552,6 +555,9 @@ class DisplayController:
# Last mode written to the display_current_state cache key, and when.
self._last_published_mode: Optional[str] = None
self._last_published_at = 0.0
# (is_display_active, on_demand_active) as last published: a change
# to either is republished at once, like a mode change.
self._last_published_flags: Optional[Tuple[bool, bool]] = None
self.global_dynamic_config = (
self.config.get("display", {}).get("dynamic_duration", {}) or {}
)
@@ -1438,14 +1444,24 @@ class DisplayController:
}
self.cache_manager.set('display_current_state', state)
self._last_published_mode = self.current_display_mode
self._last_published_flags = self._current_state_flags()
self._last_published_at = time.monotonic()
except (OSError, RuntimeError, ValueError, TypeError) as err:
logger.error("Failed to publish current display state: %s", err, exc_info=True)
def _current_state_flags(self) -> Tuple[bool, bool]:
"""The published flags besides the mode that a reader acts on."""
return (bool(self.is_display_active), bool(self.on_demand_active))
def _publish_current_mode_state_if_changed(self) -> None:
"""Publish the current mode state when it changed, or when the last
publish is older than CURRENT_STATE_REFRESH_SECONDS.
A change is the mode, or ``is_display_active`` / ``on_demand_active``:
waking from scheduled-off, or an on-demand session starting or ending,
often leaves the mode name as it was, and the web UI would otherwise
show the old flag until the next refresh.
The web UI reads this key with a max_age (api_v3/display.py), so a mode
that stays on screen longer than that -- a live game under live
priority, a single enabled plugin -- has to be republished or the UI
@@ -1453,6 +1469,7 @@ class DisplayController:
every render tick.
"""
if (self.current_display_mode != self._last_published_mode
or self._current_state_flags() != getattr(self, '_last_published_flags', None)
or time.monotonic() - self._last_published_at >= CURRENT_STATE_REFRESH_SECONDS):
self._publish_current_mode_state()
@@ -2740,11 +2757,12 @@ class DisplayController:
# into an Arbiter / ScreenRunner / Sources (docs/RUN_LOOP_REDESIGN.md).
# test/test_run_loop_golden.py pins down what the loop does with them.
def _blank_while_scheduled_off(self) -> None:
def _blank_while_scheduled_off(self, dwell: float) -> None:
"""One pass while the schedule has the panel off: blank it and dwell.
The dwell returns early when on-demand starts or the schedule turns
the panel back on (see _sleep_with_plugin_updates).
The Arbiter's SCHEDULED_OFF plan; ``dwell`` is its max_duration. The
dwell returns early when on-demand starts or the schedule turns the
panel back on (see _sleep_with_plugin_updates).
"""
# Clear display when schedule makes it inactive to ensure blank screen
# (not showing initialization screen)
@@ -2757,7 +2775,7 @@ class DisplayController:
logger.info(f"Display not active (is_display_active={self.is_display_active}), sleeping...")
self._publish_current_mode_state()
self._sleep_with_plugin_updates(60)
self._sleep_with_plugin_updates(dwell)
def _run_follower_frame(self) -> None:
"""One frame while a sync leader drives this panel (follower mode).
@@ -2836,46 +2854,69 @@ class DisplayController:
if remaining > 0:
time.sleep(remaining)
def _show_wifi_notice(self) -> bool:
"""Show a pending WiFi status message for one pass.
def _arbiter_inputs(self) -> ArbiterInputs:
"""This pass's snapshot for Arbiter.decide, taken after _evaluate_schedule.
_evaluate_schedule forces is_display_active on while an on-demand
session overrides a scheduled-off window, and flags that with
on_demand_schedule_override, so the schedule's own answer is "on and
not overridden". The WiFi notice is read only when it could win --
the panel is on and neither a follower nor on-demand outranks it --
because _check_wifi_status_message has side effects (its 1 Hz
throttle, deleting an expired or corrupt file) that such a pass
never had.
"""
schedule_on = self.is_display_active and not self.on_demand_schedule_override
on_demand = self.on_demand_active
follower = self.sync_manager.is_follower_active()
notice = None
if self.is_display_active and not follower and not on_demand:
notice = self._read_wifi_notice()
return ArbiterInputs(schedule_on=schedule_on, on_demand_active=on_demand,
follower_active=follower, wifi_notice=notice)
def _read_wifi_notice(self) -> Optional[WifiNotice]:
"""The pending WiFi notice (see _check_wifi_status_message), or None."""
status = self._check_wifi_status_message()
if not status:
return None
return WifiNotice(message=status['message'],
expires_at=float(status['expires_at']))
def _show_wifi_notice(self, notice: WifiNotice, dwell: float) -> bool:
"""Draw the Arbiter's WIFI plan and hold it for ``dwell`` seconds.
Returns True when the message was drawn, and the pass ends there
(no rotation). On-demand outranks it, so nothing is checked while
on-demand is active; a message that fails to draw is treated as no
message.
(no rotation). A message that fails to draw is treated as no
message: the pass carries on as a LEGACY plan.
"""
if self.on_demand_active:
return False
wifi_status_data = self._check_wifi_status_message()
if not wifi_status_data:
return False
self._end_scroll_before_core_screen()
if not self._display_wifi_status_message(wifi_status_data):
if not self._display_wifi_status_message(
{'message': notice.message, 'expires_at': notice.expires_at}):
# Display failed, clear the status and continue normally
return False
# The plugin that resumes afterwards must redraw
# the whole panel, not paint over the message.
self.force_change = True
self._sleep_with_plugin_updates(0.5)
self._sleep_with_plugin_updates(dwell)
return True
def _wifi_notice_pending(self) -> bool:
"""True when a WiFi notice is waiting that _show_wifi_notice would draw.
"""True when a WiFi notice should end the current screen early.
Polled from the frame loops, the dwell sleep and after a Vegas
iteration yields, so a notice preempts whatever is on the panel
within about a second instead of waiting for the screen to end --
by which time a short notice has usually expired unseen. Cheap at
frame rate: _check_wifi_status_message stats the file at most once
a second. On-demand outranks the notice, as in _show_wifi_notice.
a second. The rule is display_arbiter.wifi_notice_preempts; the
file is not read at all while on-demand, which outranks the notice,
is active.
"""
if self.on_demand_active:
return False
status = self._check_wifi_status_message()
if not status:
return False
# The 1 s throttle can hand back a result that has expired since.
return time.time() < float(status['expires_at'])
return wifi_notice_preempts(self._read_wifi_notice(), self.on_demand_active,
time.time())
def _resolve_active_mode(self):
"""The mode this pass shows: the on-demand session's current mode
@@ -3485,8 +3526,13 @@ class DisplayController:
# is active). No repaint: this screen's first frame pushes it.
self._apply_brightness_target()
if not self.is_display_active:
self._blank_while_scheduled_off()
# Who gets the panel this pass (src/display_arbiter.py). The
# Arbiter decides the scheduled-off gate, Follower and Wifi;
# a LEGACY plan carries on to the code below.
plan = Arbiter.decide(ArbiterState(), self._arbiter_inputs(), time.time())
if plan.source is Source.SCHEDULED_OFF:
self._blank_while_scheduled_off(plan.max_duration)
continue
self._publish_current_mode_state_if_changed()
@@ -3499,7 +3545,7 @@ class DisplayController:
# Multi-display sync: follower mode — render frames received from leader.
# Plugin update() threads still run (via _tick_plugin_updates above) so
# data is fresh when we return to standalone if the leader goes offline.
if self.sync_manager.is_follower_active():
if plan.source is Source.FOLLOWER:
self._run_follower_frame()
continue
@@ -3507,10 +3553,12 @@ class DisplayController:
# This also cleans up expired updates to prevent memory leaks
self.display_manager.process_deferred_updates()
# Check for WiFi status message (interrupts normal rotation, but respects on-demand)
# Priority: on-demand > wifi-status > live-priority > normal rotation
# Past this point no WiFi message is showing this pass.
if self._show_wifi_notice():
# WiFi status message: interrupts the rotation, but on-demand
# outranks it (the Arbiter's order). Past this point no WiFi
# message is showing this pass: one that failed to draw
# carries on as a LEGACY plan.
if (plan.source is Source.WIFI and plan.notice is not None
and self._show_wifi_notice(plan.notice, plan.max_duration)):
continue # Skip to next iteration, don't rotate
# Check for live priority content and switch to it immediately.
+104 -9
View File
@@ -25,6 +25,23 @@ snapshot behind. A display that stops cleanly publishes ``running: false``
on the way out, so readers see "stopped" at once rather than after the
stale window. Nothing on the reading side reports a runtime fact from a
snapshot that is not live.
Render-loop liveness. The snapshot is written from its own thread, which
keeps going when the render loop hangs inside a plugin. So the reader also
checks the render loop's heartbeat (``src/display_watchdog.py``, the file
``/api/v3/health`` reports as ``checks.display_loop``): a live snapshot from
the process whose heartbeat has gone stale is ``stalled``, as the health
check says, not ``live``. No extra writes: the heartbeat already exists, on
tmpfs. A missing heartbeat (dev server, emulator, Windows, a display still
starting up) or one from another process (a display restarted after a
watchdog kill) says nothing, and the snapshot is judged on its own.
A dead publisher. systemd removes the heartbeat's directory when the
service stops, so after a watchdog kill there is no heartbeat to go stale.
The reader then asks whether the snapshot's ``pid`` still exists (POSIX
``kill(pid, 0)``, which sends nothing): a running snapshot from a process
that is gone is ``stale`` at once rather than ``live`` for the rest of its
``stale_after`` window.
"""
import math
@@ -34,6 +51,7 @@ import time
from dataclasses import dataclass, field
from typing import Any, Callable, Dict, Optional
from src import display_watchdog
from src.logging_config import get_logger
from src.redaction import redact_credentials
@@ -70,6 +88,9 @@ _VERSION_CHARS = 40
#: Reader statuses. Only LIVE carries runtime facts.
LIVE = "live"
STALE = "stale"
#: The snapshot is fresh but the render loop's heartbeat is not: the display
#: is hung (or was just killed by the watchdog), as /api/v3/health reports.
STALLED = "stalled"
STOPPED = "stopped"
UNKNOWN = "unknown"
@@ -281,7 +302,9 @@ class PluginRuntimeView:
``status``: ``live`` (a fresh snapshot from a running display),
``stale`` (the last snapshot is older than its ``stale_after``: the
display is hung or died without cleaning up), ``stopped`` (the display
display is hung or died without cleaning up), ``stalled`` (the snapshot
is fresh but the same process's render-loop heartbeat is stale: the
render loop is hung), ``stopped`` (the display
said so on its way out) or ``unknown`` (no readable snapshot). Only a
live view reports per-plugin facts; every other status answers None for
them, so a caller cannot pass stale truth on by accident.
@@ -292,6 +315,8 @@ class PluginRuntimeView:
age_seconds: Optional[float] = None
stale_after: float = STALE_AFTER
plugins: Dict[str, Dict[str, Any]] = field(default_factory=dict)
#: Age of the render loop's heartbeat, when it was taken into account.
heartbeat_age_seconds: Optional[float] = None
@property
def live(self) -> bool:
@@ -321,6 +346,8 @@ class PluginRuntimeView:
"published_at": self.published_at,
"age_seconds": None if self.age_seconds is None else round(self.age_seconds, 1),
"stale_after": self.stale_after,
"heartbeat_age_seconds": (None if self.heartbeat_age_seconds is None
else round(self.heartbeat_age_seconds, 1)),
}
@@ -334,8 +361,56 @@ def _stale_after_of(snapshot: Dict[str, Any]) -> float:
return min(max(number, _STALE_AFTER_MIN), _STALE_AFTER_MAX)
def view_from_snapshot(snapshot: Any, now: Optional[float] = None) -> PluginRuntimeView:
"""Judge a snapshot read from the cache; never raises."""
def _heartbeat_age_for(snapshot: Dict[str, Any], heartbeat: Any,
now_mono: Optional[float]) -> Optional[float]:
"""Age of ``heartbeat`` if it comes from the process that published
``snapshot``; None when there is none, it has no time, or it belongs to
another process (a restarted display, or a heartbeat left by a killed one)."""
if not isinstance(heartbeat, dict):
return None
beat_pid = heartbeat.get("pid")
snap_pid = snapshot.get("pid")
if (isinstance(beat_pid, bool) or not isinstance(beat_pid, int)
or isinstance(snap_pid, bool) or not isinstance(snap_pid, int)
or beat_pid != snap_pid):
return None
return display_watchdog.heartbeat_age(heartbeat, now_mono=now_mono)
def process_exists(pid: int) -> Optional[bool]:
"""Whether process ``pid`` exists: True, False, or None when this
platform cannot tell. POSIX only -- on Windows ``os.kill`` terminates.
Signal 0 sends nothing; EPERM (the display runs as root, the web
interface does not) still means the process is there."""
if os.name != "posix" or pid <= 0:
return None
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
except OSError:
return None
return True
def view_from_snapshot(snapshot: Any, now: Optional[float] = None,
heartbeat: Any = None,
now_mono: Optional[float] = None,
process_alive: Optional[Callable[[int], Optional[bool]]] = None,
) -> PluginRuntimeView:
"""Judge a snapshot read from the cache; never raises.
``heartbeat`` is the render loop's heartbeat
(``display_watchdog.read_heartbeat()``), or None when there is none. A
live snapshot whose process's heartbeat is at least
``display_watchdog.HEARTBEAT_STALE_SECONDS`` old is ``stalled``: the
threshold /api/v3/health uses for ``checks.display_loop``.
``process_alive`` (``process_exists`` when reading the real cache) says
whether the snapshot's publisher still exists; a running snapshot from
one that is gone is ``stale``.
"""
if not isinstance(snapshot, dict) or snapshot.get("schema") != SNAPSHOT_SCHEMA:
return PluginRuntimeView(status=UNKNOWN)
published_at = _epoch(snapshot.get("published_at"))
@@ -351,21 +426,34 @@ def view_from_snapshot(snapshot: Any, now: Optional[float] = None) -> PluginRunt
if age > stale_after or age < -stale_after:
return PluginRuntimeView(status=STALE, published_at=published_at,
age_seconds=age, stale_after=stale_after)
pid = snapshot.get("pid")
if (process_alive is not None and isinstance(pid, int) and not isinstance(pid, bool)
and process_alive(pid) is False):
return PluginRuntimeView(status=STALE, published_at=published_at,
age_seconds=max(age, 0.0), stale_after=stale_after)
beat_age = _heartbeat_age_for(snapshot, heartbeat, now_mono)
if beat_age is not None and beat_age >= display_watchdog.HEARTBEAT_STALE_SECONDS:
return PluginRuntimeView(status=STALLED, published_at=published_at,
age_seconds=max(age, 0.0), stale_after=stale_after,
heartbeat_age_seconds=beat_age)
plugins = snapshot.get("plugins")
return PluginRuntimeView(
status=LIVE, published_at=published_at, age_seconds=max(age, 0.0),
stale_after=stale_after,
stale_after=stale_after, heartbeat_age_seconds=beat_age,
plugins={k: v for k, v in plugins.items() if isinstance(v, dict)}
if isinstance(plugins, dict) else {},
)
def read_plugin_runtime(cache_manager: Any, now: Optional[float] = None) -> PluginRuntimeView:
"""The display's latest snapshot, judged for staleness. Never raises; a
missing cache manager or an unreadable snapshot is ``unknown``.
def read_plugin_runtime(cache_manager: Any, now: Optional[float] = None,
heartbeat_path: Optional[str] = None) -> PluginRuntimeView:
"""The display's latest snapshot, judged for staleness and against the
render loop's heartbeat. Never raises; a missing cache manager or an
unreadable snapshot is ``unknown``.
memory_ttl=0: the key is written by the other process, so only the file
is current.
is current. ``heartbeat_path`` defaults to
``display_watchdog.HEARTBEAT_PATH``.
"""
if cache_manager is None:
return PluginRuntimeView(status=UNKNOWN)
@@ -374,4 +462,11 @@ def read_plugin_runtime(cache_manager: Any, now: Optional[float] = None) -> Plug
except Exception as err:
logger.debug("Could not read the plugin runtime snapshot: %s", err, exc_info=True)
return PluginRuntimeView(status=UNKNOWN)
return view_from_snapshot(snapshot, now=now)
try:
heartbeat = display_watchdog.read_heartbeat(
heartbeat_path or display_watchdog.HEARTBEAT_PATH)
except Exception as err: # read_heartbeat does not raise; belt and braces
logger.debug("Could not read the display heartbeat: %s", err, exc_info=True)
heartbeat = None
return view_from_snapshot(snapshot, now=now, heartbeat=heartbeat,
process_alive=process_exists)
+4
View File
@@ -58,6 +58,10 @@ server has none.
| `dom/test_store_dom.js` | yes | Store pagination, per-page, category, tri-state Installed button, and persistence across a re-boot, against the live registry |
| `dom/test_no_double_fetch.js` | yes | Loads the **whole** `plugins_manager.js` and counts requests: typing in the store search must filter the cached list, not refetch `/api/v3/plugins/store/list` |
| `dom/test_cache_page.js` | yes | The Cache tab as a page module (`js/pages/cache.js`) on the real partial: no inline script, one request per swap and per Refresh after repeated swaps, a cancelled request draws nothing, hostile keys stay text, delete/empty/error/login states |
| `dom/test_durations_page.js` | yes | The Rotation tab (`js/pages/durations.js`) with the real `plugin-order-list.js` widget: one plugin-list request per swap, one move per click after repeated swaps, a swap cancels the request in flight, a late widget is waited for |
| `dom/test_operation_history_page.js` | yes | The Operation History tab (`js/pages/operation-history.js`): one request per swap and per Refresh, the plugin filter filled once, paging, filters, search, Clear, error/login states, hostile values stay text |
| `dom/test_raw_json_page.js` | yes | The Config Editor tab (`js/pages/raw-json.js`): one POST per Save after repeated swaps, Format/Validate, invalid JSON never sent, a save survives a swap, the old global entry points |
| `dom/test_backup_restore_page.js` | yes | The Backup & Restore tab (`js/pages/backup-restore.js`): one request per action after repeated swaps, the upload and restore options, reads cancelled and writes not on a swap, hostile names stay text, the old global entry points |
| `dom/test_tools_sections.js` | yes | The Tools tab's MQTT bridge and Pixlet editor sections: form prefill, the write-only password (blank means unchanged), the running-session banner and countdown, and that the editor link points at the host you loaded the page from |
Point the DOM suites at a rig with a full plugin set when it matters — a dev box
+305
View File
@@ -0,0 +1,305 @@
// The Backup & Restore tab as a page module
// (static/v3/js/pages/backup-restore.js), in a real DOM (jsdom) with the real
// server-rendered partial and the real API's payload shapes. Built like
// test_cache_page.js:
//
// * the partial ships no <script> and no onclick; its root is
// data-page="backup-restore" and its buttons name an action
// * after five swaps each button makes exactly one request
// * reads in flight are cancelled by a swap; writes (export, delete,
// restore) are not, and their result is still reported
// * file names, host names, plugin ids and messages are shown as text
// * the five old globals' entry points still work
const http = require('http');
const path = require('path');
const { pathToFileURL } = require('url');
const { JSDOM, VirtualConsole } = require('jsdom');
const BASE = process.env.BASE || 'http://localhost:5000';
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
const get = p => new Promise((res, rej) =>
http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
const load = f => import(pathToFileURL(path.join(JS, f)).href);
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
let pass = 0, fail = 0;
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
(async () => {
const partial = await get('/partials/backup-restore');
const realPreview = JSON.parse(await get('/api/v3/backup/preview'));
const realList = JSON.parse(await get('/api/v3/backup/list'));
const { createRegistry } = await load('core/registry.js');
const { createApi } = await load('core/api.js');
const backupPage = await load('pages/backup-restore.js');
console.log('\n── Backup & Restore tab: page module (real DOM) ──');
ok('the partial ships no inline script', !/<script/i.test(partial));
ok('the partial has no inline handlers', !/onclick=/i.test(partial));
ok('the partial root is data-page="backup-restore"', /data-page="backup-restore"/.test(partial));
ok('its buttons name an action', ['export', 'inspect', 'restore', 'cancel', 'refresh']
.every(a => partial.includes(`data-action="${a}"`)));
ok('the real preview answers in the shape the page reads', realPreview.status === 'success'
&& ['has_config', 'has_secrets', 'has_wifi', 'user_fonts', 'plugin_uploads', 'plugins'].every(k => k in realPreview.data),
realPreview);
ok('the real list answers in the shape the page reads', realList.status === 'success' && Array.isArray(realList.data), realList);
const HOSTILE = '<img src=x onerror="window.pwned=1">';
const ZIP = 'ledmatrix-backup-host-20260930.zip';
const ODD = 'odd name #1 ' + HOSTILE + '.zip';
const preview = Object.assign({}, realPreview.data, {
has_config: true, has_secrets: false, has_wifi: true, user_fonts: ['a.bdf', HOSTILE], plugin_uploads: 3, plugins: [1, 2],
});
const listing = [
{ filename: ZIP, size: 2048, created_at: '2026-09-30 12:00:00' },
{ filename: ODD, size: 0, created_at: HOSTILE },
];
const manifest = {
created_at: '2026-09-30T12:00:00', hostname: HOSTILE, ledmatrix_version: '3.8.0',
detected_contents: ['config', 'secrets'], plugins: [{ plugin_id: 'clock' }, { plugin_id: HOSTILE }],
};
const errs = [];
let navigations = 0;
const vc = new VirtualConsole();
vc.on('jsdomError', e => {
const msg = String(e.message || e).split('\n')[0];
// jsdom does not navigate; the export's download is that navigation.
if (/Not implemented: navigation/.test(msg)) { navigations++; return; }
errs.push(msg);
});
vc.on('error', (...a) => errs.push(a.join(' ')));
const dom = new JSDOM(`<!doctype html><html><body><div id="backup-restore-content">${partial}</div></body></html>`,
{ url: BASE + '/', virtualConsole: vc });
const { window } = dom;
const doc = window.document;
const panel = doc.getElementById('backup-restore-content');
let mode = 'ok';
let restoreAnswer = null;
let aborted = 0;
const requests = [];
const pending = [];
function fakeFetch(url, init) {
requests.push({ url, method: init.method, body: init.body, signal: init.signal });
const respond = (status, body) => Promise.resolve({
status, ok: status >= 200 && status < 300,
headers: { get: () => null },
text: () => Promise.resolve(JSON.stringify(body)),
});
if (mode === 'hang') {
return new Promise((resolve, reject) => {
pending.push(() => resolve(respond(200, { status: 'success', data: [] })));
if (init.signal) init.signal.addEventListener('abort', () => {
aborted++;
const e = new Error('aborted'); e.name = 'AbortError'; reject(e);
});
});
}
if (url === '/api/v3/backup/preview') return respond(200, { status: 'success', data: preview });
if (url === '/api/v3/backup/list') return respond(200, { status: 'success', data: listing });
if (url === '/api/v3/backup/export') return respond(200, { status: 'success', filename: ZIP });
if (url === '/api/v3/backup/validate') return respond(200, { status: 'success', data: manifest });
if (url === '/api/v3/backup/restore') return restoreAnswer();
if (init.method === 'DELETE') return respond(200, { status: 'success' });
return respond(404, { status: 'error', message: 'unexpected ' + url });
}
const notes = [];
const registry = createRegistry({
document: doc,
context: { api: createApi({ fetch: fakeFetch }), notify: (m, t) => notes.push([m, t]) },
});
registry.register('backup-restore', backupPage);
const count = (url, method = 'GET') => requests.filter(r => r.url === url && r.method === method).length;
const lists = () => count('/api/v3/backup/list');
const $ = id => doc.getElementById(id);
const action = a => doc.querySelector(`button[data-action="${a}"]`);
const hidden = id => $(id).classList.contains('hidden');
async function swap() {
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
panel.innerHTML = partial;
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
await tick(20);
}
function pick(file) {
Object.defineProperty($('restore-file-input'), 'files', { value: file ? [file] : [], configurable: true });
$('restore-file-input').dispatchEvent(new window.Event('change', { bubbles: true }));
}
await registry.start();
await tick(20);
// ── first load ──────────────────────────────────────────────────────────
ok('one preview and one list request on start',
count('/api/v3/backup/preview') === 1 && lists() === 1, requests.map(r => r.url));
ok('the summary is drawn', /Main config: yes/.test($('export-preview').textContent)
&& /Secrets: no/.test($('export-preview').textContent) && /Plugin image uploads: 3 file\(s\)/.test($('export-preview').textContent),
$('export-preview').textContent);
ok('a hostile font name is text', $('export-preview').textContent.includes(HOSTILE));
const rows = () => doc.querySelectorAll('#backup-history tbody tr');
ok('one history row per backup', rows().length === 2, rows().length);
ok('a hostile file name and date are text', rows()[1].textContent.includes(ODD) && rows()[1].textContent.includes(HOSTILE));
ok('...and created no element', !doc.querySelector('#backup-restore-content img') && !window.pwned);
ok('download links encode the name', rows()[1].querySelector('a').getAttribute('href')
=== '/api/v3/backup/download/' + encodeURIComponent(ODD), rows()[1].querySelector('a').getAttribute('href'));
ok('delete buttons carry the exact name and no handler',
rows()[1].querySelector('button[data-action="delete"]').dataset.filename === ODD
&& !rows()[1].querySelector('button').getAttribute('onclick'));
// ── repeated swaps ──────────────────────────────────────────────────────
const oldRefresh = action('refresh');
for (let i = 0; i < 5; i++) await swap();
ok('one list request per swap', lists() === 6, lists());
ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
let before = lists();
action('refresh').click();
await tick(20);
ok('Refresh makes exactly one request (no duplicate listeners)', lists() === before + 1, lists() - before);
oldRefresh.click();
await tick(20);
ok('a swapped-out button does nothing', lists() === before + 1, lists() - before);
// ── delete ──────────────────────────────────────────────────────────────
let asked = null;
window.confirm = msg => { asked = msg; return false; };
rows()[1].querySelector('button[data-action="delete"]').click();
await tick(20);
ok('Delete asks first, naming the file', asked === 'Delete ' + ODD + '?', asked);
ok('cancel sends nothing', !requests.some(r => r.method === 'DELETE'));
window.confirm = () => true;
before = lists();
rows()[1].querySelector('button[data-action="delete"]').click();
await tick(20);
const deletes = requests.filter(r => r.method === 'DELETE');
ok('one DELETE request, to the encoded name', deletes.length === 1
&& deletes[0].url === '/api/v3/backup/' + encodeURIComponent(ODD), deletes.map(d => d.url));
ok('the list reloads once', lists() === before + 1, lists() - before);
ok('the deletion is reported', notes.some(n => n[0] === 'Backup deleted' && n[1] === 'success'), notes);
// ── export ──────────────────────────────────────────────────────────────
before = lists();
action('export').click();
action('export').click(); // a second click while it is busy
ok('the export button is busy while it runs', $('export-backup-btn').disabled
&& $('export-backup-btn').textContent.includes('Creating'));
await tick(30);
ok('one export request', count('/api/v3/backup/export', 'POST') === 1, count('/api/v3/backup/export', 'POST'));
ok('the new backup is reported', notes.some(n => n[0] === 'Backup created: ' + ZIP), notes);
ok('its download starts once', navigations === 1, navigations);
ok('the list reloads once', lists() === before + 1, lists() - before);
ok('the button comes back', !$('export-backup-btn').disabled
&& $('export-backup-btn').textContent.trim() === 'Download backup', $('export-backup-btn').textContent);
// ── inspect ─────────────────────────────────────────────────────────────
notes.length = 0;
action('inspect').click();
await tick(20);
ok('Inspect with no file asks for one', notes[0] && notes[0][0] === 'Choose a backup file first', notes);
ok('...and sends nothing', count('/api/v3/backup/validate', 'POST') === 0);
const file = new window.File(['PK'], 'backup.zip', { type: 'application/zip' });
pick(file);
action('inspect').click();
await tick(20);
const validate = requests.filter(r => r.url === '/api/v3/backup/validate');
ok('one validate request', validate.length === 1, validate.length);
ok('it uploads the file as backup_file', validate[0] && validate[0].body instanceof window.FormData
&& validate[0].body.get('backup_file').name === 'backup.zip');
ok('the contents are shown', !hidden('restore-preview')
&& /Includes: config, secrets/.test($('restore-preview-body').textContent), $('restore-preview-body').textContent);
ok('a hostile host name and plugin id are text', $('restore-preview-body').textContent.includes(HOSTILE)
&& !doc.querySelector('#restore-preview-body img'));
pick(new window.File(['PK2'], 'other.zip'));
ok('picking another file hides the old contents', hidden('restore-preview'));
notes.length = 0;
window.confirm = () => true;
action('restore').click();
await tick(20);
ok('Restore needs an inspected file', notes[0] && notes[0][0] === 'Inspect the file before restoring'
&& count('/api/v3/backup/restore', 'POST') === 0, notes);
// ── restore ─────────────────────────────────────────────────────────────
pick(file);
action('inspect').click();
await tick(20);
$('opt-secrets').checked = false;
restoreAnswer = () => Promise.resolve({
status: 200, ok: true, headers: { get: () => null },
text: () => Promise.resolve(JSON.stringify({ status: 'success', data: {
success: true, restored: ['config', HOSTILE], skipped: ['secrets'], plugins_installed: [], plugins_failed: [], errors: [],
} })),
});
window.confirm = () => false;
action('restore').click();
await tick(20);
ok('Restore asks first', count('/api/v3/backup/restore', 'POST') === 0);
window.confirm = () => true;
notes.length = 0;
action('restore').click();
await tick(30);
const restores = requests.filter(r => r.url === '/api/v3/backup/restore');
ok('one restore request', restores.length === 1, restores.length);
const options = restores[0] && JSON.parse(restores[0].body.get('options'));
ok('it sends exactly the six options, as booleans', options && Object.keys(options).sort().join() ===
'reinstall_plugins,restore_config,restore_fonts,restore_plugin_uploads,restore_secrets,restore_wifi'
&& options.restore_secrets === false && options.restore_config === true, options);
ok('...with the inspected file', restores[0] && restores[0].body.get('backup_file').name === 'backup.zip');
ok('a restore is not cancelled by a swap', restores[0] && restores[0].signal === undefined);
ok('the result is drawn', !hidden('restore-result') && /Restore complete/.test($('restore-result').textContent)
&& /Skipped: secrets/.test($('restore-result').textContent) && /Restart the display service/.test($('restore-result').textContent),
$('restore-result').textContent);
ok('a hostile restored name is text', $('restore-result').textContent.includes(HOSTILE) && !doc.querySelector('#restore-result img'));
ok('...in green, reported once', $('restore-result').classList.contains('bg-green-50')
&& notes.length === 1 && notes[0][0] === 'Restore complete' && notes[0][1] === 'success', notes);
ok('the restore button comes back', !$('run-restore-btn').disabled && $('run-restore-btn').textContent.trim() === 'Restore now');
restoreAnswer = () => Promise.resolve({
status: 500, ok: false, headers: { get: () => null },
text: () => Promise.resolve(JSON.stringify({ status: 'error', message: 'Restore incomplete — failed: secrets',
data: { errors: ['secrets'] } })),
});
notes.length = 0;
action('restore').click();
await tick(30);
ok('a failed restore reports the server\'s message', notes[0] && notes[0][0] === 'Restore failed: Restore incomplete — failed: secrets'
&& notes[0][1] === 'error', notes);
action('cancel').click();
ok('Cancel hides the contents and forgets the file', hidden('restore-preview') && hidden('restore-result')
&& $('restore-file-input').value === '');
notes.length = 0;
action('restore').click();
await tick(20);
ok('...so Restore asks for an inspection again', notes[0] && notes[0][0] === 'Inspect the file before restoring', notes);
// ── reads in flight are cancelled by a swap ─────────────────────────────
mode = 'hang';
action('refresh').click();
await tick(5);
mode = 'ok';
await swap();
ok('the swap cancelled the list request', aborted === 1, aborted);
pending.forEach(resolve => resolve());
await tick(20);
ok('the new page drew its own list', rows().length === 2, rows().length);
// ── the old globals ─────────────────────────────────────────────────────
before = lists();
await backupPage.loadBackupList();
ok('loadBackupList() reloads once', lists() === before + 1, lists() - before);
notes.length = 0;
await backupPage.validateRestoreFile();
ok('validateRestoreFile() inspects', notes[0] && notes[0][0] === 'Choose a backup file first', notes);
const exportsBefore = count('/api/v3/backup/export', 'POST');
await backupPage.exportBackup();
ok('exportBackup() exports once', count('/api/v3/backup/export', 'POST') === exportsBefore + 1);
ok('clearRestore and runRestore are there', typeof backupPage.clearRestore === 'function'
&& typeof backupPage.runRestore === 'function');
ok('no DOM errors', errs.length === 0, errs);
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
+178
View File
@@ -0,0 +1,178 @@
// The Rotation & Durations tab as a page module
// (static/v3/js/pages/durations.js), in a real DOM (jsdom) with the real
// server-rendered partial, the real plugin-order-list widget and the real
// /api/v3/plugins/installed payload shape. Built like test_cache_page.js:
//
// * the partial ships no <script>; its root is data-page="durations"
// * the rotation list loads once per swap-in, however many swaps came first
// * a list request still in flight when the page is swapped away is
// cancelled and draws nothing
// * a widget that loads late is waited for, and a page swapped away while
// waiting starts nothing
// * plugin names reach the page as text
const http = require('http');
const fs = require('fs');
const path = require('path');
const { pathToFileURL } = require('url');
const { JSDOM, VirtualConsole } = require('jsdom');
const BASE = process.env.BASE || 'http://localhost:5000';
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
const get = p => new Promise((res, rej) =>
http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
const load = f => import(pathToFileURL(path.join(JS, f)).href);
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
let pass = 0, fail = 0;
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
(async () => {
const partial = await get('/partials/durations');
const real = JSON.parse(await get('/api/v3/plugins/installed'));
const { createRegistry } = await load('core/registry.js');
const { createApi } = await load('core/api.js');
const durationsPage = await load('pages/durations.js');
const widgetSource = fs.readFileSync(path.join(JS, 'widgets/plugin-order-list.js'), 'utf8');
console.log('\n── Rotation & Durations tab: page module (real DOM) ──');
ok('the partial ships no inline script', !/<script/i.test(partial));
ok('the partial root is data-page="durations"', /data-page="durations"/.test(partial));
ok('the rotation list and its hidden input are in the partial',
/id="rotation_plugin_order"/.test(partial) && /id="rotation_plugin_order_value"/.test(partial));
ok('the real API answers in the shape the widget reads',
real.status === 'success' && real.data && Array.isArray(real.data.plugins), real);
const HOSTILE = '<img src=x onerror="window.pwned=1">';
const plugins = [
{ id: 'clock', name: 'Clock', enabled: true },
{ id: 'weather', name: HOSTILE, enabled: true },
{ id: 'stocks', name: 'Stocks', enabled: true },
{ id: 'off', name: 'Disabled one', enabled: false },
];
const errs = [];
const vc = new VirtualConsole();
vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
vc.on('error', (...a) => errs.push(a.join(' ')));
const dom = new JSDOM(`<!doctype html><html><body><div id="durations-content">${partial}</div></body></html>`,
{ url: BASE + '/', virtualConsole: vc, runScripts: 'outside-only' });
const { window } = dom;
const doc = window.document;
const panel = doc.getElementById('durations-content');
let mode = 'ok';
const requests = [];
const pending = [];
let aborted = 0;
function fakeFetch(url, init = {}) {
requests.push({ url, method: init.method || 'GET' });
const respond = (status, body) => Promise.resolve({
status, ok: status >= 200 && status < 300,
headers: { get: () => null },
json: () => Promise.resolve(body),
text: () => Promise.resolve(JSON.stringify(body)),
});
if (mode === 'hang') {
return new Promise((resolve, reject) => {
pending.push(resolve);
if (init.signal) init.signal.addEventListener('abort', () => {
aborted++;
const e = new Error('aborted'); e.name = 'AbortError'; reject(e);
});
});
}
return respond(200, { status: 'success', data: { plugins } });
}
// The widget is a classic script: it calls the window's own fetch.
window.fetch = fakeFetch;
window.eval(widgetSource);
const widget = window.PluginOrderList;
ok('the widget script defines PluginOrderList', !!(widget && widget.init));
const registry = createRegistry({
document: doc,
context: { api: createApi({ fetch: fakeFetch }), notify: () => {} },
});
registry.register('durations', durationsPage);
const lists = () => requests.filter(r => r.url === '/api/v3/plugins/installed').length;
const $ = id => doc.getElementById(id);
const order = () => JSON.parse($('rotation_plugin_order_value').value || '[]');
async function swap() {
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
panel.innerHTML = partial;
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
await tick(20);
}
await registry.start();
await tick(20);
// ── first load ──────────────────────────────────────────────────────────
ok('one plugin-list request on start', lists() === 1, lists());
let rows = doc.querySelectorAll('#rotation_plugin_order .plugin-order-item');
ok('one row per enabled plugin', rows.length === 3, rows.length);
ok('the hidden input holds the order', order().length === 3 && order().includes('clock'), order());
ok('a hostile plugin name is shown as text', $('rotation_plugin_order').textContent.includes(HOSTILE));
ok('...and created no element', !doc.querySelector('#rotation_plugin_order img') && !window.pwned);
// ── repeated swaps ──────────────────────────────────────────────────────
for (let i = 0; i < 5; i++) await swap();
ok('one plugin-list request per swap', lists() === 6, lists());
ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
rows = doc.querySelectorAll('#rotation_plugin_order .plugin-order-item');
ok('the list is drawn once, not stacked', rows.length === 3, rows.length);
const before = order();
const down = rows[0].querySelector('button[aria-label^="Move"][aria-label$="down"]');
down.click();
await tick(5);
const after = order();
ok('Move down moves one place (one listener)',
after[0] === before[1] && after[1] === before[0] && after[2] === before[2], [before, after]);
ok('reordering makes no request', lists() === 6, lists());
// ── in flight when swapped away ─────────────────────────────────────────
mode = 'hang';
await swap();
ok('a request is in flight', pending.length === 1, pending.length);
mode = 'ok';
await swap();
await tick(20);
ok('the new page drew its own list', doc.querySelectorAll('#rotation_plugin_order .plugin-order-item').length === 3);
ok('the cancelled request drew no error', !/Error loading plugins/.test($('rotation_plugin_order').textContent));
ok('the swap cancelled the request (ctx.signal reaches the widget)', aborted === 1, aborted);
// Answer it late anyway, with one plugin: the new page's list is untouched.
pending[0]({ status: 200, ok: true, headers: { get: () => null },
json: () => Promise.resolve({ status: 'success', data: { plugins: [plugins[0]] } }) });
await tick(20);
ok('a late answer to the cancelled request redraws nothing',
doc.querySelectorAll('#rotation_plugin_order .plugin-order-item').length === 3,
doc.querySelectorAll('#rotation_plugin_order .plugin-order-item').length);
// ── the widget loads late ───────────────────────────────────────────────
delete window.PluginOrderList;
const beforeLate = lists();
await swap();
ok('no request while the widget is missing', lists() === beforeLate, lists() - beforeLate);
window.PluginOrderList = widget;
await tick(150);
ok('starts once the widget arrives', lists() === beforeLate + 1, lists() - beforeLate);
delete window.PluginOrderList;
await swap();
const beforeGone = lists();
// Swapped to another tab's content while still waiting for the widget.
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
panel.innerHTML = '<p>another tab</p>';
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
window.PluginOrderList = widget;
await tick(250);
ok('a page swapped away while waiting starts nothing', lists() === beforeGone, lists() - beforeGone);
ok('nothing left mounted', registry.list().length === 0, registry.list().length);
ok('no DOM errors', errs.length === 0, errs);
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
+231
View File
@@ -0,0 +1,231 @@
// The Operation History tab as a page module
// (static/v3/js/pages/operation-history.js), in a real DOM (jsdom) with the
// real server-rendered partial and the real API's payload shape. Built like
// test_cache_page.js:
//
// * the partial ships no <script>; its root is data-page="operation-history"
// * one history request per swap-in, and after five swaps Refresh, Clear,
// Next and the filters each act exactly once
// * the plugin filter is filled once per swap-in, not once per swap so far
// * a request in flight when the page is swapped away is cancelled
// * plugin ids, users and error messages from the log are shown as text
const http = require('http');
const path = require('path');
const { pathToFileURL } = require('url');
const { JSDOM, VirtualConsole } = require('jsdom');
const BASE = process.env.BASE || 'http://localhost:5000';
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
const get = p => new Promise((res, rej) =>
http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
const load = f => import(pathToFileURL(path.join(JS, f)).href);
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
let pass = 0, fail = 0;
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
(async () => {
const partial = await get('/partials/operation-history');
const real = JSON.parse(await get('/api/v3/plugins/operation/history?limit=1000'));
const { createRegistry } = await load('core/registry.js');
const { createApi } = await load('core/api.js');
const historyPage = await load('pages/operation-history.js');
console.log('\n── Operation History tab: page module (real DOM) ──');
ok('the partial ships no inline script', !/<script/i.test(partial));
ok('the partial root is data-page="operation-history"', /data-page="operation-history"/.test(partial));
ok('the real API answers in the shape the page reads',
real.status === 'success' && Array.isArray(real.data), real);
const realKeys = real.data.length ? Object.keys(real.data[0]) : null;
ok('a real record has the fields the page reads',
!realKeys || ['operation_type', 'plugin_id', 'status', 'timestamp'].every(k => realKeys.includes(k)), realKeys);
const HOSTILE = '<img src=x onerror="window.pwned=1">';
function record(i, extra) {
return Object.assign({
operation_id: 'op' + i, operation_type: ['install', 'update', 'enable'][i % 3],
plugin_id: i % 2 ? 'clock' : 'weather', status: i % 4 === 0 ? 'error' : 'completed',
user: 'web', timestamp: 1790000000 * 1000 - i * 60000, details: { version: '1.' + i },
}, extra || {});
}
let records = [
record(0, { plugin_id: HOSTILE, error: HOSTILE + ' went wrong', user: HOSTILE,
details: { commit: 'abcdef1234', previous_commit: '1234567890' } }),
].concat(Array.from({ length: 119 }, (_, i) => record(i + 1)));
const errs = [];
const vc = new VirtualConsole();
vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
vc.on('error', (...a) => errs.push(a.join(' ')));
const dom = new JSDOM(`<!doctype html><html><body><div id="operation-history-content">${partial}</div></body></html>`,
{ url: BASE + '/', virtualConsole: vc });
const { window } = dom;
const doc = window.document;
const panel = doc.getElementById('operation-history-content');
let mode = 'ok';
let aborted = 0;
const requests = [];
function fakeFetch(url, init) {
requests.push({ url, method: init.method });
const respond = (status, body, headers = {}) => Promise.resolve({
status, ok: status >= 200 && status < 300,
headers: { get: n => headers[n] || null },
text: () => Promise.resolve(JSON.stringify(body)),
});
if (url === '/api/v3/plugins/installed') {
return respond(200, { status: 'success', data: { plugins: [{ id: 'clock' }, { id: 'weather' }, { id: HOSTILE }] } });
}
if (init.method === 'DELETE') return respond(200, { status: 'success', message: 'Operation history cleared' });
if (mode === 'network') return Promise.reject(new TypeError('Failed to fetch'));
if (mode === 'error') return respond(500, { status: 'error', message: 'Operation history not initialized' });
if (mode === 'login') return respond(401, { status: 'error' }, { 'X-LEDMatrix-Login': '/login' });
if (mode === 'hang') {
return new Promise((resolve, reject) => {
init.signal.addEventListener('abort', () => {
aborted++;
const e = new Error('aborted'); e.name = 'AbortError'; reject(e);
});
});
}
return respond(200, { status: 'success', data: records });
}
const notes = [];
const registry = createRegistry({
document: doc,
context: { api: createApi({ fetch: fakeFetch }), notify: (m, t) => notes.push([m, t]) },
});
registry.register('operation-history', historyPage);
const count = (url, method = 'GET') => requests.filter(r => r.url === url && r.method === method).length;
const HISTORY = '/api/v3/plugins/operation/history?limit=1000';
const histories = () => count(HISTORY);
const $ = id => doc.getElementById(id);
const rows = () => doc.querySelectorAll('#history-table-body tr');
const showing = () => [$('history-start').textContent, $('history-end').textContent, $('history-total').textContent].join('/');
async function swap() {
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
panel.innerHTML = partial;
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
await tick(20);
}
function change(id, value) {
$(id).value = value;
$(id).dispatchEvent(new window.Event('change', { bubbles: true }));
}
await registry.start();
await tick(20);
// ── first load ──────────────────────────────────────────────────────────
ok('one history request on start', histories() === 1, histories());
ok('one plugin-list request on start', count('/api/v3/plugins/installed') === 1);
ok('the first page holds 50 rows', rows().length === 50, rows().length);
ok('the counters say 1 to 50 of 120', showing() === '1/50/120', showing());
ok('Previous is off on the first page', $('history-prev-btn').disabled && !$('history-next-btn').disabled);
const first = rows()[0];
ok('a hostile plugin id, user and error are shown as text',
first.textContent.includes(HOSTILE) && first.textContent.includes(HOSTILE + ' went wrong'));
ok('...and created no element', !doc.querySelector('#history-table-body img') && !window.pwned);
ok('a hostile plugin id in the filter is text too',
[...$('history-plugin-filter').options].some(o => o.value === HOSTILE && o.textContent === HOSTILE));
ok('details are summarised', first.textContent.includes('commit: abcdef1') && first.textContent.includes('from: 1234567'));
ok('"error" is shown as failed, in red',
first.querySelectorAll('span')[1].textContent === 'failed' && first.querySelector('.bg-red-100'));
// ── repeated swaps ──────────────────────────────────────────────────────
const oldRefresh = $('refresh-history-btn');
for (let i = 0; i < 5; i++) await swap();
ok('one history request per swap', histories() === 6, histories());
ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
ok('the plugin filter is filled once', $('history-plugin-filter').options.length === 4,
$('history-plugin-filter').options.length);
let before = histories();
$('refresh-history-btn').click();
await tick(20);
ok('Refresh makes exactly one request (no duplicate listeners)', histories() === before + 1, histories() - before);
oldRefresh.click();
await tick(20);
ok('a swapped-out button does nothing', histories() === before + 1, histories() - before);
$('history-next-btn').click();
ok('Next moves one page', showing() === '51/100/120', showing());
$('history-next-btn').click();
ok('...and the last page is short', showing() === '101/120/120' && rows().length === 20, showing());
ok('Next is off on the last page', $('history-next-btn').disabled);
$('refresh-history-btn').click();
await tick(20);
ok('Refresh keeps the page', showing() === '101/120/120', showing());
$('history-prev-btn').click();
ok('Previous moves one page', showing() === '51/100/120', showing());
// ── filters ─────────────────────────────────────────────────────────────
change('history-status-filter', 'failed');
ok('the status filter matches "error" records as failed', showing() === '1/30/30', showing());
change('history-status-filter', '');
change('history-plugin-filter', 'weather');
ok('the plugin filter', [...rows()].every(r => r.children[2].textContent === 'weather') && showing() === '1/50/59', showing());
change('history-plugin-filter', '');
$('history-search').value = 'no such thing';
$('history-search').dispatchEvent(new window.Event('input', { bubbles: true }));
ok('search waits for the typing to stop', rows().length === 50);
await tick(350);
ok('a search with no match says so', rows().length === 1 && rows()[0].textContent.includes('No operations found'));
ok('...and the counters follow', showing() === '0/0/0', showing());
$('history-search').value = '';
$('history-search').dispatchEvent(new window.Event('input', { bubbles: true }));
await swap(); // swapped away with the search still pending
await tick(350);
ok('a pending search on a swapped-out page does nothing', errs.length === 0, errs);
// ── clear ───────────────────────────────────────────────────────────────
let asked = null;
window.confirm = msg => { asked = msg; return false; };
$('clear-history-btn').click();
await tick(20);
ok('Clear asks first', asked && /clear the operation history/.test(asked), asked);
ok('cancel sends nothing', count('/api/v3/plugins/operation/history', 'DELETE') === 0);
window.confirm = () => true;
$('clear-history-btn').click();
await tick(20);
ok('one DELETE request', count('/api/v3/plugins/operation/history', 'DELETE') === 1,
count('/api/v3/plugins/operation/history', 'DELETE'));
ok('the table empties', rows().length === 1 && showing() === '0/0/0', showing());
// ── states ──────────────────────────────────────────────────────────────
notes.length = 0;
mode = 'error';
$('refresh-history-btn').click(); await tick(20);
ok('an API error is reported', notes.length === 1 && notes[0][0] === 'Failed to load operation history' && notes[0][1] === 'error', notes);
mode = 'network';
$('refresh-history-btn').click(); await tick(20);
ok('a network failure is reported', notes.length === 2 && notes[1][0] === 'Error loading operation history', notes);
mode = 'login';
$('refresh-history-btn').click(); await tick(20);
ok('the login redirect reports nothing', notes.length === 2, notes);
// ── in flight when swapped away ─────────────────────────────────────────
mode = 'hang';
$('refresh-history-btn').click(); await tick(5);
mode = 'ok';
await swap();
ok('the swap cancelled the request', aborted === 1, aborted);
ok('the new page drew its own list', rows().length === 50 && notes.length === 2, [rows().length, notes]);
// ── PluginAPI's cached installed list, when it is loaded ────────────────
let pluginApiCalls = 0;
window.PluginAPI = { getInstalledPlugins: () => { pluginApiCalls++; return Promise.resolve([{ id: 'cached' }]); } };
const directBefore = count('/api/v3/plugins/installed');
await swap();
ok('the plugin filter uses PluginAPI when it is there', pluginApiCalls === 1
&& count('/api/v3/plugins/installed') === directBefore, [pluginApiCalls, count('/api/v3/plugins/installed') - directBefore]);
ok('...and is filled from it', [...$('history-plugin-filter').options].map(o => o.value).join() === ',cached');
delete window.PluginAPI;
ok('no DOM errors', errs.length === 0, errs);
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
+187
View File
@@ -0,0 +1,187 @@
// The Config Editor tab as a page module (static/v3/js/pages/raw-json.js), in
// a real DOM (jsdom) with the real server-rendered partial and the real
// endpoints' answers. Built like test_cache_page.js:
//
// * the partial ships no <script> and no onclick; its root is
// data-page="raw-json" and its buttons name an action and an editor
// * after five swaps, each Save makes exactly one request and each
// Format / Validate acts once
// * invalid JSON is never sent, and the parser's message is shown as text
// * a save is a write: a swap does not cancel it, and its result is still
// reported
// * the old globals' entry points still work
const http = require('http');
const path = require('path');
const { pathToFileURL } = require('url');
const { JSDOM, VirtualConsole } = require('jsdom');
const BASE = process.env.BASE || 'http://localhost:5000';
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
const get = p => new Promise((res, rej) =>
http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
const load = f => import(pathToFileURL(path.join(JS, f)).href);
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
let pass = 0, fail = 0;
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
(async () => {
const partial = await get('/partials/raw-json');
const { createRegistry } = await load('core/registry.js');
const { createApi } = await load('core/api.js');
const rawPage = await load('pages/raw-json.js');
console.log('\n── Config Editor tab: page module (real DOM) ──');
ok('the partial ships no inline script', !/<script/i.test(partial));
ok('the partial has no inline handlers', !/onclick=/i.test(partial));
ok('the partial root is data-page="raw-json"', /data-page="raw-json"/.test(partial));
ok('six buttons name an action and an editor',
(partial.match(/data-action="(format|validate|save)" data-editor="(main|secrets)"/g) || []).length === 6);
const errs = [];
const vc = new VirtualConsole();
vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
vc.on('error', (...a) => errs.push(a.join(' ')));
const dom = new JSDOM(`<!doctype html><html><body><div id="config-editor-content">${partial}</div></body></html>`,
{ url: BASE + '/', virtualConsole: vc });
const { window } = dom;
const doc = window.document;
const panel = doc.getElementById('config-editor-content');
let mode = 'ok';
const requests = [];
const pending = [];
function fakeFetch(url, init) {
requests.push({ url, method: init.method, body: init.body, signal: init.signal });
const respond = (status, body) => Promise.resolve({
status, ok: status >= 200 && status < 300,
headers: { get: () => null },
text: () => Promise.resolve(JSON.stringify(body)),
});
if (mode === 'network') return Promise.reject(new TypeError('Failed to fetch'));
if (mode === 'error') return respond(400, { status: 'error', message: 'Configuration must be a JSON object' });
if (mode === 'hang') return new Promise(resolve => pending.push(() => resolve(respond(200, { status: 'success' }))));
return respond(200, { status: 'success', message: 'Main configuration saved successfully' });
}
const notes = [];
const registry = createRegistry({
document: doc,
context: { api: createApi({ fetch: fakeFetch }), notify: (m, t) => notes.push([m, t]) },
});
registry.register('raw-json', rawPage);
const $ = id => doc.getElementById(id);
const button = (action, editor) => doc.querySelector(`button[data-action="${action}"][data-editor="${editor}"]`);
const posts = url => requests.filter(r => r.url === url && r.method === 'POST');
async function swap() {
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
panel.innerHTML = partial;
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
await tick(20);
}
function type(id, text) {
$(id).value = text;
$(id).dispatchEvent(new window.Event('input', { bubbles: true }));
}
await registry.start();
await tick(20);
// ── first load ──────────────────────────────────────────────────────────
ok('the real files parse', (() => { try { JSON.parse($('main-config-editor').value); JSON.parse($('secrets-config-editor').value); return true; } catch (e) { return false; } })());
ok('both editors are validated on start',
$('main-config-validation').textContent.trim() === 'Valid JSON' && $('secrets-config-validation').textContent.trim() === 'Valid JSON',
[$('main-config-validation').textContent, $('secrets-config-validation').textContent]);
ok('nothing is requested on start', requests.length === 0, requests.length);
// ── repeated swaps ──────────────────────────────────────────────────────
const oldSave = button('save', 'main');
for (let i = 0; i < 5; i++) await swap();
ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
type('main-config-editor', '{"display": {"brightness": 50}, "note": "a\\"b"}');
button('save', 'main').click();
await tick(20);
ok('Save makes exactly one request (no duplicate listeners)', posts('/api/v3/config/raw/main').length === 1,
posts('/api/v3/config/raw/main').length);
const sent = posts('/api/v3/config/raw/main')[0];
ok('it posts the parsed file as JSON', sent && JSON.parse(sent.body).note === 'a"b' && JSON.parse(sent.body).display.brightness === 50);
ok('success is reported once', notes.length === 1 && notes[0][0] === 'config.json saved successfully!' && notes[0][1] === 'success', notes);
oldSave.click();
await tick(20);
ok('a swapped-out button does nothing', posts('/api/v3/config/raw/main').length === 1);
button('save', 'secrets').click();
await tick(20);
ok('Save on the secrets editor posts to the secrets endpoint, once', posts('/api/v3/config/raw/secrets').length === 1);
ok('...and names that file', notes[1] && notes[1][0] === 'config_secrets.json saved successfully!', notes);
// ── format and validate ─────────────────────────────────────────────────
notes.length = 0;
type('main-config-editor', '{"a":1,"b":[1,2]}');
button('format', 'main').click();
ok('Format re-indents by four spaces', $('main-config-editor').value === '{\n "a": 1,\n "b": [\n 1,\n 2\n ]\n}',
$('main-config-editor').value);
ok('...and says so once', notes.length === 1 && notes[0][0] === 'JSON formatted successfully!', notes);
button('validate', 'main').click();
ok('Validate shows the detailed box', $('main-config-validation').textContent.includes('JSON is valid!'));
ok('...and says so once', notes.length === 2 && notes[1][0] === 'JSON validation successful!', notes);
// ── invalid JSON ────────────────────────────────────────────────────────
const HOSTILE = '{"x": <img src=x onerror="window.pwned=1">';
type('main-config-editor', HOSTILE);
ok('typing re-validates', /^Invalid JSON: /.test($('main-config-validation').textContent.trim()),
$('main-config-validation').textContent);
ok('the parser message is text, not markup', !doc.querySelector('#main-config-validation img') && !window.pwned);
notes.length = 0;
const postsBefore = requests.length;
button('save', 'main').click();
await tick(20);
ok('invalid JSON is not sent', requests.length === postsBefore);
ok('...and the user is told', notes.length === 1 && notes[0][0] === 'Invalid JSON! Please fix errors before saving.', notes);
button('format', 'main').click();
ok('Format refuses invalid JSON', $('main-config-editor').value === HOSTILE && /^Cannot format invalid JSON/.test(notes[1][0]), notes);
button('validate', 'main').click();
ok('Validate shows the error box', $('main-config-validation').textContent.includes('Invalid JSON syntax')
&& !doc.querySelector('#main-config-validation img'));
// ── server answers ──────────────────────────────────────────────────────
type('main-config-editor', '{"a": 1}');
notes.length = 0;
mode = 'error';
button('save', 'main').click(); await tick(20);
ok('a refused save shows the server\'s message', notes[0] && notes[0][0] === 'Error saving config.json: Configuration must be a JSON object', notes);
mode = 'network';
button('save', 'main').click(); await tick(20);
ok('a network failure says so', notes[1] && notes[1][0] === 'Error saving config.json: Failed to fetch', notes);
// ── a save is a write: a swap does not cancel it ────────────────────────
mode = 'hang';
notes.length = 0;
button('save', 'main').click(); await tick(5);
const inFlight = requests[requests.length - 1];
ok('the save carries no page signal', inFlight.signal === undefined);
mode = 'ok';
await swap();
pending.forEach(resolve => resolve());
await tick(20);
ok('its result is still reported after the swap', notes.length === 1 && notes[0][1] === 'success', notes);
// ── the old globals ─────────────────────────────────────────────────────
type('main-config-editor', '[1,2]');
ok('validateJSON(editorId) answers synchronously', rawPage.validateJSON('main-config-editor') === true);
type('main-config-editor', '[1,');
ok('...false for invalid JSON', rawPage.validateJSON('main-config-editor') === false);
type('secrets-config-editor', '{"k":"v"}');
rawPage.formatJson('secrets-config-editor', 'secrets-config-validation');
ok('formatJson(editorId, validationId) formats that editor', $('secrets-config-editor').value === '{\n "k": "v"\n}');
const before = posts('/api/v3/config/raw/secrets').length;
await rawPage.saveSecretsConfig();
ok('saveSecretsConfig() saves once', posts('/api/v3/config/raw/secrets').length === before + 1);
ok('no DOM errors', errs.length === 0, errs);
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
+3 -1
View File
@@ -25,7 +25,9 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
'unit/test_store_registry_fields.js', 'unit/test_restart_banner.js',
'unit/test_page_registry.js', 'unit/test_core_modules.js'];
const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
'dom/test_tools_sections.js', 'dom/test_cache_page.js'];
'dom/test_tools_sections.js', 'dom/test_cache_page.js',
'dom/test_durations_page.js', 'dom/test_operation_history_page.js',
'dom/test_raw_json_page.js', 'dom/test_backup_restore_page.js'];
function reachable(url) {
return new Promise(res => {
+4
View File
@@ -51,6 +51,10 @@ async function rejection(promise) {
const controller = new AbortController();
await api.get('/api/v3/x', { signal: controller.signal });
ok('the signal is passed to fetch', calls[2][1].signal === controller.signal);
const upload = { kind: 'form-data' };
await api.request('POST', '/api/v3/backup/validate', { body: upload });
ok('a raw body (an upload) is sent as it is, with no JSON Content-Type',
calls[3][1].body === upload && calls[3][1].headers['Content-Type'] === undefined, calls[3][1].headers);
// Default: window.fetch looked up per call, so base.html's login wrapper
// (installed before any module runs, or replaced later) is the one used.
+3 -6
View File
@@ -105,18 +105,15 @@ const ESCAPERS = [
'static/v3/js/widgets/slider.js', 'function escapeAttr(text) {', 'escapeAttr', false],
['display.html (escapeAttr)',
'templates/v3/partials/display.html', 'function escapeAttr(text) {', 'escapeAttr', false],
['backup_restore.html (escapeHtml)',
'templates/v3/partials/backup_restore.html', 'function escapeHtml(value) {', 'escapeHtml', false],
['operation_history.html (escapeHtml)',
'templates/v3/partials/operation_history.html', 'function escapeHtml(text) {', 'escapeHtml', false],
['tools.html (escHtml)',
'templates/v3/partials/tools.html', 'function escHtml(s) {', 'escHtml', false],
['tools.html (phEscape)',
'templates/v3/partials/tools.html', 'function phEscape(s) {', 'phEscape', false],
['logs.html (escapeHtml)',
'templates/v3/partials/logs.html', 'function escapeHtml(text) {', 'escapeHtml', false],
// cache.html has no script any more: js/pages/cache.js builds its rows with
// textContent, and test/js/dom/test_cache_page.js checks a hostile key.
// cache.html, backup_restore.html and operation_history.html have no
// script any more: their js/pages/ modules draw server data with
// textContent, and each page's suite in test/js/dom/ checks a hostile value.
];
// The breakout payload: closes a double-quoted attribute and opens an event
+36 -11
View File
@@ -205,27 +205,52 @@ class TestCacheLifetimeWithRealCacheManager:
class TestEspnHelpers:
@freeze_time('2026-08-07')
def test_fetch_espn_scoreboard_url_params_and_cache_key(self, helper):
helper.get = Mock(return_value={'ok': 1})
def test_fetch_espn_scoreboard_url_params_and_cache_key(self, helper, cache):
# The canonical key (fetch service stage 2), shared with every other
# consumer of the scoreboard; the old key is only read as a fallback.
cache.get_cached_data.return_value = None
helper._get = Mock(return_value={'ok': 1})
result = helper.fetch_espn_scoreboard('football', 'nfl')
assert result == {'ok': 1}
helper.get.assert_called_once_with(
helper._get.assert_called_once_with(
'https://site.api.espn.com/apis/site/v2/sports/football/nfl/scoreboard',
params={'dates': '20260807', 'limit': ESPN_MAX_LIMIT},
cache_key='espn_football_nfl_20260807',
cache_ttl=300,
{'dates': '20260807', 'limit': ESPN_MAX_LIMIT},
None, None, None, 300, 300,
)
read = [c.args[0] for c in cache.get_cached_data.call_args_list]
assert read == ['espn_scoreboard_football_nfl_20260807', 'espn_football_nfl_20260807']
cache.set.assert_called_once_with('espn_scoreboard_football_nfl_20260807', {'ok': 1})
def test_fetch_espn_scoreboard_explicit_date(self, helper):
helper.get = Mock(return_value=None)
def test_fetch_espn_scoreboard_explicit_date(self, helper, cache):
cache.get_cached_data.return_value = None
helper._get = Mock(return_value=None)
helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115')
kwargs = helper.get.call_args.kwargs
assert kwargs['params'] == {'dates': '20250115', 'limit': ESPN_MAX_LIMIT}
assert kwargs['cache_key'] == 'espn_basketball_nba_20250115'
args = helper._get.call_args.args
assert args[1] == {'dates': '20250115', 'limit': ESPN_MAX_LIMIT}
cache.set.assert_not_called() # a failed fetch caches nothing
def test_fetch_espn_scoreboard_an_explicit_key_works_as_before(self, helper):
helper.get = Mock(return_value={'ok': 1})
helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115', cache_key='mine')
assert helper.get.call_args.kwargs['cache_key'] == 'mine'
def test_fetch_espn_scoreboard_reads_the_old_key_after_an_upgrade(self, helper, cache):
import time as _time
records = {'espn_basketball_nba_20250115': {
'timestamp': _time.time() - 10, 'ttl': 300, 'data': {'events': ['old']}}}
cache.get_cached_data.side_effect = lambda key, **kw: records.get(key)
helper._get = Mock()
assert helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115') == {
'events': ['old']}
helper._get.assert_not_called()
def test_fetch_espn_standings_url_and_cache_key(self, helper):
helper.get = Mock(return_value={'ok': 1})
+239
View File
@@ -0,0 +1,239 @@
"""Arbiter.decide (src/display_arbiter.py): the stage-2 Sources as tables.
decide() is pure, so every case is one row of (inputs) -> expected Source.
The rows are written out, not computed, so a change to the priority order
has to change a row here as well. The last class checks the controller's
side: the snapshot run() gathers gives the same scheduled-off answer as
is_display_active did before the Arbiter, through an on-demand session
that overrides the schedule and ends (#714).
"""
import itertools
import os
from unittest.mock import MagicMock, patch
import pytest
os.environ.setdefault("EMULATOR", "true")
from src import display_arbiter # noqa: E402
from src.display_arbiter import ( # noqa: E402
SCHEDULED_OFF_DWELL,
WIFI_NOTICE_DWELL,
Arbiter,
ArbiterInputs,
ArbiterState,
ScreenPlan,
Source,
WifiNotice,
wifi_notice_preempts,
)
NOTICE = WifiNotice(message="Connected to HomeNet", expires_at=1_000.0)
OFF = Source.SCHEDULED_OFF
FOLLOW = Source.FOLLOWER
WIFI = Source.WIFI
LEGACY = Source.LEGACY
# (schedule_on, on_demand_active, follower_active, notice) -> Source.
# Every combination of the stage-2 inputs: 2 x 2 x 2 x 2 = 16 rows.
DECIDE_TABLE = [
# The gate: scheduled off and no on-demand session -- blank, whatever
# else is pending, a follower and a WiFi notice included.
(False, False, False, None, OFF),
(False, False, False, NOTICE, OFF),
(False, False, True, None, OFF),
(False, False, True, NOTICE, OFF),
# Scheduled off, but on-demand overrides the gate.
(False, True, False, None, LEGACY), # on-demand: run() decides
(False, True, False, NOTICE, LEGACY), # on-demand outranks WiFi
(False, True, True, None, FOLLOW), # follower outranks on-demand
(False, True, True, NOTICE, FOLLOW),
# Scheduled on.
(True, False, False, None, LEGACY), # live / Vegas / rotation
(True, False, False, NOTICE, WIFI),
(True, False, True, None, FOLLOW),
(True, False, True, NOTICE, FOLLOW), # follower outranks WiFi
(True, True, False, None, LEGACY),
(True, True, False, NOTICE, LEGACY), # on-demand outranks WiFi
(True, True, True, None, FOLLOW),
(True, True, True, NOTICE, FOLLOW),
]
def _inputs(schedule_on, on_demand, follower, notice):
return ArbiterInputs(schedule_on=schedule_on, on_demand_active=on_demand,
follower_active=follower, wifi_notice=notice)
def test_the_table_covers_every_combination_once():
keys = [row[:4] for row in DECIDE_TABLE]
combos = list(itertools.product([False, True], [False, True],
[False, True], [None, NOTICE]))
assert sorted(keys, key=repr) == sorted(combos, key=repr)
class TestDecide:
@pytest.mark.parametrize("schedule_on,on_demand,follower,notice,expected",
DECIDE_TABLE)
def test_source(self, schedule_on, on_demand, follower, notice, expected):
plan = Arbiter.decide(ArbiterState(),
_inputs(schedule_on, on_demand, follower, notice),
500.0)
assert plan.source is expected
@pytest.mark.parametrize("schedule_on,on_demand,follower,notice,expected",
DECIDE_TABLE)
def test_plan_fields(self, schedule_on, on_demand, follower, notice, expected):
plan = Arbiter.decide(ArbiterState(),
_inputs(schedule_on, on_demand, follower, notice),
500.0)
if expected is OFF:
assert plan == ScreenPlan(OFF, max_duration=SCHEDULED_OFF_DWELL)
elif expected is WIFI:
assert plan == ScreenPlan(WIFI, max_duration=WIFI_NOTICE_DWELL,
notice=NOTICE)
else:
# A follower paces itself; LEGACY is run()'s existing code.
assert plan == ScreenPlan(expected)
def test_dwells_are_todays(self):
# The constants that run() used to hard-code.
assert SCHEDULED_OFF_DWELL == 60.0
assert WIFI_NOTICE_DWELL == 0.5
@pytest.mark.parametrize("now", [0.0, NOTICE.expires_at - 1,
NOTICE.expires_at, NOTICE.expires_at + 1e6])
def test_top_of_pass_wifi_takes_the_notice_as_read(self, now):
"""Before the Arbiter, the top of the pass drew whatever notice
_check_wifi_status_message returned without comparing the expiry;
decide() must not start comparing it with ``now``."""
plan = Arbiter.decide(ArbiterState(), _inputs(True, False, False, NOTICE), now)
assert plan.source is WIFI
class TestPurity:
def test_decide_reads_no_clock(self):
boom = MagicMock(side_effect=AssertionError("decide read the clock"))
with patch("time.time", boom), patch("time.monotonic", boom), \
patch("time.perf_counter", boom):
for *key, expected in DECIDE_TABLE:
assert Arbiter.decide(ArbiterState(), _inputs(*key), 1.0).source is expected
def test_the_module_imports_no_io_or_clock(self):
for name in ("time", "os", "datetime", "threading", "json", "pathlib"):
assert not hasattr(display_arbiter, name), name
def test_same_inputs_same_plan(self):
for *key, _expected in DECIDE_TABLE:
inputs = _inputs(*key)
first = Arbiter.decide(ArbiterState(), inputs, 1.0)
assert Arbiter.decide(ArbiterState(), inputs, 1.0) == first
assert inputs == _inputs(*key) # not mutated
def test_inputs_and_plans_are_frozen(self):
inputs = _inputs(True, False, False, NOTICE)
with pytest.raises(Exception):
inputs.schedule_on = False # type: ignore[misc]
plan = Arbiter.decide(ArbiterState(), inputs, 1.0)
with pytest.raises(Exception):
plan.source = OFF # type: ignore[misc]
# (notice, on_demand_active, now) -> preempts. The mid-screen rule.
PREEMPT_TABLE = [
(None, False, 0.0, False),
(None, True, 0.0, False),
(NOTICE, False, NOTICE.expires_at - 0.001, True),
(NOTICE, False, NOTICE.expires_at, False), # expired at expires_at
(NOTICE, False, NOTICE.expires_at + 5, False), # the throttle's stale copy
(NOTICE, True, NOTICE.expires_at - 0.001, False), # on-demand outranks it
(NOTICE, True, NOTICE.expires_at + 5, False),
]
@pytest.mark.parametrize("notice,on_demand,now,expected", PREEMPT_TABLE)
def test_wifi_notice_preempts(notice, on_demand, now, expected):
assert wifi_notice_preempts(notice, on_demand, now) is expected
class TestControllerSnapshot:
"""DisplayController._arbiter_inputs, on a stub controller."""
def _controller(self, *, display_active=True, override=False, on_demand=False,
follower=False, status=None):
from src.display_controller import DisplayController
dc = object.__new__(DisplayController)
dc.is_display_active = display_active
dc.on_demand_schedule_override = override
dc.on_demand_active = on_demand
dc.sync_manager = MagicMock()
dc.sync_manager.is_follower_active.return_value = follower
dc._check_wifi_status_message = MagicMock(return_value=status)
return dc
def test_reads_the_notice_when_it_can_win(self):
dc = self._controller(status={"message": "AP mode", "expires_at": 12,
"timestamp": 7, "duration": 5})
inputs = dc._arbiter_inputs()
assert inputs == ArbiterInputs(schedule_on=True, on_demand_active=False,
follower_active=False,
wifi_notice=WifiNotice("AP mode", 12.0))
@pytest.mark.parametrize("kwargs", [
{"display_active": False},
{"follower": True},
{"on_demand": True, "override": True},
{"on_demand": True},
])
def test_does_not_read_the_notice_when_it_cannot_win(self, kwargs):
"""Reading it deletes an expired file and moves the 1 Hz throttle,
which these passes never did before the Arbiter."""
dc = self._controller(status={"message": "x", "expires_at": 1e12}, **kwargs)
assert dc._arbiter_inputs().wifi_notice is None
dc._check_wifi_status_message.assert_not_called()
def test_an_on_demand_override_is_not_the_schedule(self):
dc = self._controller(display_active=True, override=True, on_demand=True)
assert dc._arbiter_inputs().schedule_on is False
def test_the_gate_matches_is_display_active_through_an_on_demand_session(self):
"""Through _evaluate_schedule, as run() calls it: the Arbiter blanks
exactly when is_display_active is False, which is what run() tested
before. The schedule is 07:00-23:00."""
from test.test_display_controller_schedule import at, make_controller
dc = make_controller({"schedule": {"enabled": True, "start_time": "07:00",
"end_time": "23:00"},
"timezone": "UTC"})
dc.on_demand_active = False
dc.on_demand_schedule_override = False
dc.sync_manager = MagicMock()
dc.sync_manager.is_follower_active.return_value = False
dc._check_wifi_status_message = MagicMock(return_value=None)
def step(time_str):
p = at(time_str)
try:
dc._evaluate_schedule()
finally:
p.stop()
plan = Arbiter.decide(ArbiterState(), dc._arbiter_inputs(), 0.0)
assert (plan.source is OFF) is (not dc.is_display_active), time_str
return plan.source
assert step("22:59:30") is LEGACY
assert step("23:00:00") is OFF # window ends
dc.on_demand_active = True
assert step("23:00:10") is LEGACY # on-demand overrides
assert dc.on_demand_schedule_override is True
assert step("23:01:00") is LEGACY # next minute, still on
dc._reset_on_demand_fields() # session ends
assert step("23:01:20") is OFF # same minute: blanks
dc.on_demand_active = True
assert step("06:59:00") is LEGACY
assert step("07:00:00") is LEGACY # schedule back on mid-session
dc._reset_on_demand_fields()
assert step("07:00:30") is LEGACY
@@ -3,6 +3,8 @@
- systemd stops the service with SIGTERM; it must run the same cleanup as Ctrl-C.
- The current mode is republished while it stays on screen, so the web UI's
"Now showing" does not turn into "unknown" after its 120 s max_age.
- Waking from scheduled-off, blanking, or on-demand starting or ending with
the mode name unchanged is republished at once, not up to 30 s later.
- Switching Vegas on in the web UI works when Vegas was off at startup.
"""
@@ -78,6 +80,67 @@ def test_unchanged_mode_is_republished_after_the_refresh_interval():
assert dc.cache_manager.set.call_count == 2
def _published(dc):
return dc.cache_manager.set.call_args[0][1]
@pytest.mark.parametrize("attr,before,after", [
("is_display_active", False, True), # woke from scheduled-off
("is_display_active", True, False), # blanked for scheduled-off
("on_demand_active", False, True), # on-demand started on this mode
("on_demand_active", True, False), # ...and ended on it
])
def test_a_flag_change_with_the_same_mode_is_republished_at_once(attr, before, after):
"""ledpi: current-status showed is_display_active stale for up to 30 s
after a wake from scheduled-off, because the mode name had not changed."""
dc = _publisher()
setattr(dc, attr, before)
now = [1000.0]
with patch.object(dc_module.time, "monotonic", lambda: now[0]):
dc._publish_current_mode_state() # e.g. the blank's publish
assert _published(dc)[attr] is before
now[0] += 1 # well inside the refresh
setattr(dc, attr, after)
dc._publish_current_mode_state_if_changed() # the next publish point
assert dc.cache_manager.set.call_count == 2
assert _published(dc)[attr] is after
assert _published(dc)["mode"] == "mlb_live"
def test_wake_inside_the_blank_sleep_is_published_by_the_service_helper():
"""The blank sleeps through _service_pending_changes; the schedule turning
the display back on there reaches the web UI on that same call."""
dc = _publisher()
dc.is_display_active = False
dc._last_pending_service = None
dc.PENDING_CHANGES_INTERVAL = 0.25
for name in ("_poll_on_demand_requests", "_check_on_demand_expiration",
"_apply_brightness_target"):
setattr(dc, name, MagicMock())
def wake():
dc.is_display_active = True
dc._evaluate_schedule = MagicMock(side_effect=wake)
now = [5000.0]
with patch.object(dc_module.time, "monotonic", lambda: now[0]):
dc._publish_current_mode_state() # published while blank
now[0] += 2
dc._service_pending_changes()
assert dc.cache_manager.set.call_count == 2
assert _published(dc)["is_display_active"] is True
def test_unchanged_flags_do_not_add_writes():
dc = _publisher()
now = [1000.0]
with patch.object(dc_module.time, "monotonic", lambda: now[0]):
dc._publish_current_mode_state_if_changed()
for _ in range(5):
now[0] += 1
dc._publish_current_mode_state_if_changed()
assert dc.cache_manager.set.call_count == 1
def test_refresh_interval_is_well_inside_the_web_max_age():
# api_v3/display.py reads display_current_state with max_age=120.
assert dc_module.CURRENT_STATE_REFRESH_SECONDS < 120 / 2
@@ -176,4 +239,4 @@ def test_pending_vegas_init_is_applied_by_the_helper_the_main_loop_calls():
def test_main_loop_applies_pending_vegas_init_before_the_follower_branch():
import inspect
src = inspect.getsource(DisplayController.run)
assert src.index("self._apply_pending_vegas_init()") < src.index("self.sync_manager.is_follower_active()")
assert src.index("self._apply_pending_vegas_init()") < src.index("self._run_follower_frame()")
+493
View File
@@ -0,0 +1,493 @@
"""One cache key per ESPN scoreboard (fetch service stage 2).
- every core caller names a scoreboard with the same key;
- the keys each consumer used before are read as a fallback;
- a shared entry is never returned older than the reader's own max_age,
whoever wrote it and whatever ttl they stored (checked against a real
CacheManager, whose stored ttl otherwise wins over the reader's);
- two consumers of one scoreboard cost one request;
- the counters see it.
No network: sessions are fakes, and the fetch service is a fresh one per test.
"""
import json
import logging
import threading
import time
from datetime import date, datetime
import pytest
import requests
from requests.structures import CaseInsensitiveDict
from src.common import espn_dates
from src.common import fetch_service as fs
from src.common.espn_dates import (
ESPN_MAX_LIMIT,
espn_scoreboard_cache_key,
espn_scoreboard_cache_key_for_url,
espn_scoreboard_url,
get_espn_scoreboard,
read_espn_scoreboard_cache,
store_espn_scoreboard_cache,
)
from src.common.fetch_service import FetchService, plugin_scope
from src.common.sports_fetch import SportsFetchMixin
NFL_URL = "https://site.api.espn.com/apis/site/v2/sports/football/nfl/scoreboard"
# --- fakes ---------------------------------------------------------------------------
class Clock:
def __init__(self):
self.t = 1000.0
def now(self):
return self.t
def _response(body, url, max_age=None):
response = requests.Response()
response.status_code = 200
response._content = json.dumps(body).encode()
response.headers = CaseInsensitiveDict(
{"Cache-Control": f"max-age={max_age}"} if max_age else {})
response.url = url
response.encoding = "utf-8"
return response
class Espn(requests.Session):
"""A Session answering every scoreboard GET with ``{"events": [<dates>]}``."""
def __init__(self, max_age=None, fail=None):
super().__init__()
self.calls = []
self.max_age = max_age
self.fail = fail
self._lock = threading.Lock()
def get(self, url, **kwargs):
with self._lock:
self.calls.append((url, dict(kwargs.get("params") or {})))
if self.fail:
raise self.fail
dates = (kwargs.get("params") or {}).get("dates", "current")
return _response({"events": [{"id": dates}]}, url, self.max_age)
class RecordCache:
"""The CacheManager surface these helpers use, records and all."""
def __init__(self):
self.records = {}
self.deleted = []
def set(self, key, data, ttl=None):
record = {"timestamp": time.time()}
if ttl is not None:
record["ttl"] = ttl
record["data"] = data
self.records[key] = record
def put(self, key, data, age, ttl=None):
self.set(key, data, ttl)
self.records[key]["timestamp"] -= age
def get_cached_data(self, key, max_age=300, memory_ttl=None):
return self.records.get(key)
def get(self, key, max_age=300, memory_ttl=None):
record = self.records.get(key)
return record["data"] if record else None
def delete(self, key):
self.deleted.append(key)
self.records.pop(key, None)
@pytest.fixture
def service(monkeypatch):
svc = FetchService({"rate_limits": {}})
monkeypatch.setattr(fs, "_service", svc)
return svc
def _totals(svc):
return svc.snapshot()["totals"]
# --- the key -------------------------------------------------------------------------
class TestCanonicalKey:
def test_one_spelling_for_one_scoreboard(self):
key = "espn_scoreboard_football_nfl_20261004"
assert espn_scoreboard_cache_key("football", "nfl", "20261004") == key
assert espn_scoreboard_cache_key(" Football ", "NFL", "20261004") == key
assert espn_scoreboard_cache_key("football", "nfl", date(2026, 10, 4)) == key
assert espn_scoreboard_cache_key("football", "nfl", datetime(2026, 10, 4, 23, 59)) == key
assert espn_scoreboard_cache_key("football", "nfl", ("20261004", date(2026, 10, 4))) == key
assert espn_scoreboard_cache_key_for_url(NFL_URL, "20261004") == key
assert espn_scoreboard_cache_key_for_url(NFL_URL + "?limit=500", "20261004") == key
@pytest.mark.parametrize("dates,suffix", [
(None, "current"),
("", "current"),
("2026", "2026"),
("202610", "202610"),
("20260925-20261016", "20260925-20261016"),
((date(2026, 9, 25), date(2026, 10, 16)), "20260925-20261016"),
])
def test_every_dates_form_espn_takes(self, dates, suffix):
assert espn_scoreboard_cache_key("soccer", "eng.1", dates) == f"espn_scoreboard_soccer_eng.1_{suffix}"
@pytest.mark.parametrize("sport,league,dates", [
("football", "nfl", "2026-10-04"),
("football", "nfl", "tomorrow"),
("football", "nfl", ("20261004",)),
("football", "nfl", True),
("football", "", "20261004"),
("foot ball", "nfl", "20261004"),
("football", "nfl/../x", "20261004"),
])
def test_anything_else_is_refused_not_guessed(self, sport, league, dates):
with pytest.raises(ValueError):
espn_scoreboard_cache_key(sport, league, dates)
def test_a_url_that_is_not_a_scoreboard_has_no_key(self):
assert espn_scoreboard_cache_key_for_url(
"https://site.api.espn.com/apis/site/v2/sports/football/nfl/teams") is None
def test_the_url_it_names(self):
assert espn_scoreboard_url("Football", "college-football") == (
"https://site.api.espn.com/apis/site/v2/sports/football/college-football/scoreboard")
class Host(SportsFetchMixin):
def __init__(self, session, cache):
self.session = session
self.headers = {"User-Agent": "test"}
self.cache_manager = cache
self.logger = logging.getLogger("test.espn_cache")
self._games_lock = threading.RLock()
self.sport = "football"
self.league = "nfl"
class TestEveryCallerUsesTheKey:
"""The core helpers that cache a scoreboard all write the same key."""
DAY = "20261004"
KEY = "espn_scoreboard_football_nfl_20261004"
def test_get_espn_scoreboard(self, service):
cache = RecordCache()
get_espn_scoreboard(Espn(), "football", "nfl", self.DAY, cache_manager=cache)
assert list(cache.records) == [self.KEY]
def test_api_helper(self, service):
from src.common.api_helper import APIHelper
cache = RecordCache()
helper = APIHelper(cache_manager=cache)
helper.set_rate_limit(0)
helper.session = Espn()
helper.fetch_espn_scoreboard("football", "nfl", date=self.DAY)
assert list(cache.records) == [self.KEY]
def test_the_scoreboard_mixin(self, service):
cache = RecordCache()
host = Host(Espn(), cache)
assert host._schedule_cache_key(self.DAY) == self.KEY
host._fetch_season_directly(NFL_URL, self.DAY, None, "today")
assert list(cache.records) == [self.KEY]
def test_the_mixin_without_a_scoreboard_url(self, service):
cache = RecordCache()
Host(Espn(), cache)._fetch_season_directly(
"https://example.test/feed", self.DAY, None, "today")
assert list(cache.records) == [self.KEY]
def test_an_explicit_key_is_still_the_callers(self, service):
cache = RecordCache()
Host(Espn(), cache)._fetch_season_directly(NFL_URL, self.DAY, "mine", "today")
assert list(cache.records) == ["mine"]
# --- reading ---------------------------------------------------------------------------
class TestRead:
KEY = "espn_scoreboard_football_nfl_20261004"
OLD = "scoreboard_data_football_nfl_20261004"
def test_a_fresh_canonical_entry(self, service):
cache = RecordCache()
cache.put(self.KEY, {"events": ["new"]}, age=10)
assert read_espn_scoreboard_cache(cache, self.KEY, 30) == {"events": ["new"]}
assert (_totals(service)["cache_hits"], _totals(service)["legacy_cache_hits"]) == (1, 0)
def test_the_old_key_is_read_after_an_upgrade(self, service):
cache = RecordCache()
cache.put(self.OLD, {"events": ["old"]}, age=10)
assert read_espn_scoreboard_cache(cache, self.KEY, 30, [self.OLD]) == {"events": ["old"]}
assert (_totals(service)["cache_hits"], _totals(service)["legacy_cache_hits"]) == (1, 1)
def test_the_canonical_key_wins_over_an_old_one(self, service):
cache = RecordCache()
cache.put(self.OLD, {"events": ["old"]}, age=1)
cache.put(self.KEY, {"events": ["new"]}, age=20)
assert read_espn_scoreboard_cache(cache, self.KEY, 30, [self.OLD]) == {"events": ["new"]}
def test_a_stale_old_key_is_a_miss_too(self, service):
cache = RecordCache()
cache.put(self.OLD, {"events": ["old"]}, age=31)
assert read_espn_scoreboard_cache(cache, self.KEY, 30, [self.OLD]) is None
assert _totals(service)["cache_hits"] == 0
def test_age_is_judged_against_an_injected_clock(self, service):
cache = RecordCache()
cache.records[self.KEY] = {"timestamp": 5000.0, "data": {"events": []}}
assert read_espn_scoreboard_cache(cache, self.KEY, 30, now=5030.0) == {"events": []}
assert read_espn_scoreboard_cache(cache, self.KEY, 30, now=5030.5) is None
def test_never_older_than_the_readers_ttl_whatever_the_writer_stored(self, service):
# A writer that stored ttl=3600 must not make a 30 s reader take an
# entry a minute old (CacheManager itself would let the ttl win).
cache = RecordCache()
cache.put(self.KEY, {"events": []}, age=60, ttl=3600)
assert read_espn_scoreboard_cache(cache, self.KEY, 30) is None
assert read_espn_scoreboard_cache(cache, self.KEY, 90) == {"events": []}
def test_accept_can_turn_down_an_entry_and_sees_its_age(self, service):
cache = RecordCache()
cache.records[self.KEY] = {"timestamp": 5000.0, "data": {"events": ["live"]}}
seen = []
def live_needs_30s(data, age):
seen.append(age)
return age <= 30
assert read_espn_scoreboard_cache(cache, self.KEY, 300, now=5040.0,
accept=live_needs_30s) is None
assert read_espn_scoreboard_cache(cache, self.KEY, 300, now=5020.0,
accept=live_needs_30s) == {"events": ["live"]}
assert seen == [40.0, 20.0]
assert _totals(service)["cache_hits"] == 1 # the turned-down read is no hit
@pytest.mark.parametrize("max_age", [0, -5])
def test_no_max_age_no_read(self, service, max_age):
cache = RecordCache()
cache.put(self.KEY, {"events": []}, age=0)
assert read_espn_scoreboard_cache(cache, self.KEY, max_age) is None
def test_a_broken_cache_is_a_miss(self, service):
class Broken:
def get_cached_data(self, *a, **k):
raise OSError("disk gone")
assert read_espn_scoreboard_cache(Broken(), self.KEY, 30) is None
assert read_espn_scoreboard_cache(None, self.KEY, 30) is None
def test_a_cache_without_records(self, service):
class Plain:
def get(self, key, max_age=None):
return {"events": [key, max_age]}
assert read_espn_scoreboard_cache(Plain(), self.KEY, 29.5) == {"events": [self.KEY, 30]}
class TestWithARealCacheManager:
"""The same promise against CacheManager, memory and disk tiers."""
KEY = "espn_scoreboard_football_nfl_20261004"
@pytest.fixture
def cm(self, tmp_path):
from src.cache.disk_cache import DiskCache
from src.cache.memory_cache import MemoryCache
from src.cache_manager import CacheManager
cm = CacheManager()
cm._disk_cache_component = DiskCache(cache_dir=str(tmp_path))
cm._memory_cache_component = MemoryCache()
return cm
def _age_on_disk(self, cm, seconds):
path = cm._disk_cache_component.get_cache_path(self.KEY)
with open(path, encoding="utf-8") as fh:
record = json.load(fh)
record["timestamp"] = time.time() - seconds
with open(path, "w", encoding="utf-8") as fh:
json.dump(record, fh)
cm._memory_cache_component.clear()
def test_a_writers_long_ttl_does_not_outlast_the_readers(self, cm, service):
cm.set(self.KEY, {"events": [1]}, ttl=3600)
self._age_on_disk(cm, 60)
assert cm.get(self.KEY, max_age=30) == {"events": [1]} # what a plain read does
assert read_espn_scoreboard_cache(cm, self.KEY, 30) is None
assert read_espn_scoreboard_cache(cm, self.KEY, 120) == {"events": [1]}
def test_a_copy_loaded_into_memory_keeps_its_real_age(self, cm, service):
store_espn_scoreboard_cache(cm, self.KEY, {"events": [1]})
self._age_on_disk(cm, 60)
assert read_espn_scoreboard_cache(cm, self.KEY, 120) == {"events": [1]} # now in memory
assert read_espn_scoreboard_cache(cm, self.KEY, 30) is None
def test_the_canonical_entry_stores_no_ttl(self, cm, service):
store_espn_scoreboard_cache(cm, self.KEY, {"events": []})
path = cm._disk_cache_component.get_cache_path(self.KEY)
with open(path, encoding="utf-8") as fh:
assert "ttl" not in json.load(fh)
# --- fetching through the cache ----------------------------------------------------------------
class TestGetEspnScoreboard:
def test_a_miss_fetches_a_whole_page_and_caches_it(self, service):
cache, espn = RecordCache(), Espn()
data = get_espn_scoreboard(espn, "football", "nfl", "20261004", cache_manager=cache)
assert data == {"events": [{"id": "20261004"}]}
assert espn.calls == [(NFL_URL, {"dates": "20261004", "limit": ESPN_MAX_LIMIT})]
assert "ttl" not in cache.records["espn_scoreboard_football_nfl_20261004"]
def test_a_hit_sends_nothing(self, service):
cache, espn = RecordCache(), Espn()
get_espn_scoreboard(espn, "football", "nfl", "20261004", cache_manager=cache)
get_espn_scoreboard(espn, "football", "nfl", "20261004", cache_manager=cache, max_age=60)
assert len(espn.calls) == 1
assert _totals(service)["cache_hits"] == 1
def test_max_age_zero_always_fetches_and_still_caches(self, service):
cache, espn = RecordCache(), Espn()
get_espn_scoreboard(espn, "football", "nfl", "20261004", cache_manager=cache, max_age=0)
get_espn_scoreboard(espn, "football", "nfl", "20261004", cache_manager=cache, max_age=0)
assert len(espn.calls) == 2
assert "espn_scoreboard_football_nfl_20261004" in cache.records
def test_the_current_scoreboard_sends_no_dates(self, service):
espn = Espn()
get_espn_scoreboard(espn, "football", "nfl", cache_manager=RecordCache())
assert espn.calls == [(NFL_URL, {"limit": ESPN_MAX_LIMIT})]
def test_a_range_is_fetched_in_chunks_once_espn_rejects_it(self, service, monkeypatch):
monkeypatch.setattr(espn_dates, "_ranges_rejected_until", time.monotonic() + 60)
cache, espn = RecordCache(), Espn()
data = get_espn_scoreboard(espn, "football", "nfl", "20261003-20261004", cache_manager=cache)
assert [e["id"] for e in data["events"]] == ["20261003", "20261004"]
assert list(cache.records) == ["espn_scoreboard_football_nfl_20261003-20261004"]
def test_a_failure_raises_and_caches_nothing(self, service):
cache = RecordCache()
with pytest.raises(requests.ConnectionError):
get_espn_scoreboard(Espn(fail=requests.ConnectionError("down")), "football", "nfl",
"20261004", cache_manager=cache)
assert cache.records == {}
def test_no_cache_manager_just_fetches(self, service):
espn = Espn()
get_espn_scoreboard(espn, "football", "nfl", "20261004")
get_espn_scoreboard(espn, "football", "nfl", "20261004")
assert len(espn.calls) == 2
def test_the_response_cache_is_bounded_by_the_callers_ttl(self, monkeypatch):
clock = Clock()
svc = FetchService({"rate_limits": {}}, clock=clock.now)
monkeypatch.setattr(fs, "_service", svc)
espn = Espn(max_age=450)
get_espn_scoreboard(espn, "football", "nfl", "20261004", max_age=0)
clock.t += 20
get_espn_scoreboard(espn, "football", "nfl", "20261004", max_age=10)
assert len(espn.calls) == 2 # 20 s old > the caller's 10
clock.t += 5
get_espn_scoreboard(espn, "football", "nfl", "20261004", max_age=10)
assert len(espn.calls) == 2 # 5 s old, and ESPN said 450
assert svc.snapshot()["totals"]["memo_hits"] == 1
class TestTwoConsumersOneFetch:
"""A scoreboard's live poll and odds-ticker asking for the same day."""
def test_the_second_consumer_is_served_from_the_first_ones_copy(self, service):
cache, espn = RecordCache(), Espn()
with plugin_scope("football-scoreboard"):
live = get_espn_scoreboard(espn, "football", "nfl", "20261004",
cache_manager=cache, max_age=0)
with plugin_scope("odds-ticker"):
ticker = get_espn_scoreboard(
espn, "football", "nfl", "20261004", cache_manager=cache, max_age=300,
legacy_keys=["scoreboard_data_football_nfl_20261004"])
assert ticker == live
assert len(espn.calls) == 1
snap = service.snapshot()
assert snap["plugins"]["odds-ticker"]["cache_hits"] == 1
assert snap["plugins"]["odds-ticker"]["requests"] == 0
assert snap["plugins"]["football-scoreboard"]["requests"] == 1
def test_concurrent_misses_go_to_espn_once(self, service):
gate = threading.Event()
class Slow(Espn):
def get(self, url, **kwargs):
gate.wait(5)
return super().get(url, **kwargs)
espn, results = Slow(), []
threads = [threading.Thread(target=lambda: results.append(get_espn_scoreboard(
espn, "football", "nfl", "20261004", cache_manager=RecordCache(), max_age=60)))
for _ in range(2)]
for thread in threads:
thread.start()
deadline = time.monotonic() + 5
while time.monotonic() < deadline:
with service._lock:
if any(f.waiters for f in service._inflight.values()):
break
time.sleep(0.005)
gate.set()
for thread in threads:
thread.join(5)
assert len(espn.calls) == 1 and results[0] == results[1]
# --- the scoreboards' schedule window ----------------------------------------------------------
class TestCachedSchedule:
KEY = "espn_scoreboard_football_nfl_20260925-20261016"
YESTERDAY = "espn_scoreboard_football_nfl_20260924-20261015"
OLD = "nfl_schedule_window_7_14"
def test_the_canonical_copy(self, service):
cache = RecordCache()
cache.put(self.KEY, {"events": ["new"]}, age=0)
assert Host(Espn(), cache)._cached_schedule(self.KEY, [self.OLD]) == {"events": ["new"]}
assert cache.deleted == []
def test_the_old_key_after_an_upgrade_counted_as_legacy(self, service):
cache = RecordCache()
cache.put(self.OLD, {"events": ["old"]}, age=0)
assert Host(Espn(), cache)._cached_schedule(self.KEY, [self.OLD]) == {"events": ["old"]}
totals = _totals(service)
assert (totals["legacy_cache_hits"], totals["cache_hits"]) == (1, 0)
def test_a_miss_retires_yesterdays_window(self, service):
cache = RecordCache()
cache.put(self.YESTERDAY, {"events": []}, age=0)
assert Host(Espn(), cache)._cached_schedule(self.KEY) is None
assert cache.deleted == [self.YESTERDAY]
assert self.YESTERDAY not in cache.records
def test_a_key_that_is_not_a_window_retires_nothing(self, service):
cache = RecordCache()
Host(Espn(), cache)._cached_schedule("espn_scoreboard_football_nfl_20261004")
Host(Espn(), cache)._cached_schedule(self.OLD)
assert cache.deleted == []
def test_a_cache_without_delete(self, service):
class NoDelete(RecordCache):
delete = None
assert Host(Espn(), NoDelete())._cached_schedule(self.KEY) is None
+264
View File
@@ -876,3 +876,267 @@ def test_the_web_route_returns_the_published_counters(clock):
assert body["status"] == "success"
assert body["data"]["status"] == "live"
assert body["data"]["data"]["plugins"]["weather"]["requests"] == 1
# --- response cache (stage 2: Cache-Control max-age) -------------------------------------------
def fresh_for(seconds, body=b'{"ok": 1}', **headers):
"""A handler answering 200 with ``Cache-Control: max-age=<seconds>``."""
def handler(url, kwargs):
return make_response(200, body, url=url, headers={
"Cache-Control": f"max-age={seconds}", **headers})
return handler
class NeverHits:
"""A cache that always misses, so APIHelper always reaches the network."""
def get(self, key, max_age=None):
return None
def set(self, key, value, ttl=None):
pass
class TestResponseCache:
def test_an_identical_get_inside_max_age_is_not_sent(self, service):
session = FakeSession(fresh_for(60))
first = service.get(session, "https://api.test/x", params={"d": 1}, timeout=5)
second = service.get(session, "https://api.test/x", params={"d": 1}, timeout=5)
assert len(session.calls) == 1
assert second.json() == first.json() == {"ok": 1}
assert second is not first
totals = _counters(service)
assert (totals["requests"], totals["memo_hits"]) == (1, 1)
def test_a_hit_is_a_copy_the_caller_may_change(self, service):
session = FakeSession(fresh_for(60))
service.get(session, "https://api.test/x").headers["X-Mine"] = "1"
assert "X-Mine" not in service.get(session, "https://api.test/x").headers
def test_past_max_age_the_network_is_asked_again(self, service, clock):
session = FakeSession(fresh_for(10))
service.get(session, "https://api.test/x")
clock.advance(9.9)
service.get(session, "https://api.test/x")
assert len(session.calls) == 1
clock.advance(0.2)
service.get(session, "https://api.test/x")
assert len(session.calls) == 2
def test_the_age_header_shortens_the_lifetime(self, service, clock):
session = FakeSession(fresh_for(10, Age="8"))
service.get(session, "https://api.test/x")
clock.advance(2.5)
service.get(session, "https://api.test/x")
assert len(session.calls) == 2
def test_never_older_than_the_callers_own_ttl(self, service, clock):
session = FakeSession(fresh_for(400))
service.get(session, "https://api.test/x", cache_max_age=0)
clock.advance(20)
service.get(session, "https://api.test/x", cache_max_age=10) # 20 s > 10
assert len(session.calls) == 2
clock.advance(20)
service.get(session, "https://api.test/x", cache_max_age=30) # 20 s <= 30
assert len(session.calls) == 2
service.get(session, "https://api.test/x", cache_max_age=19.9) # 20 s > 19.9
assert len(session.calls) == 3
def test_a_caller_that_does_not_say_gets_the_default_limit(self, service, clock):
session = FakeSession(fresh_for(450))
service.get(session, "https://api.test/x")
clock.advance(service.default_max_age - 0.5)
service.get(session, "https://api.test/x")
assert len(session.calls) == 1
clock.advance(1)
service.get(session, "https://api.test/x")
assert len(session.calls) == 2
# ...while a caller with a longer TTL still takes the server at its word.
clock.advance(100)
service.get(session, "https://api.test/x", cache_max_age=300)
assert len(session.calls) == 2
def test_zero_always_asks_but_still_fills_the_cache(self, service):
session = FakeSession(fresh_for(60))
service.get(session, "https://api.test/x", cache_max_age=0)
service.get(session, "https://api.test/x", cache_max_age=0)
assert len(session.calls) == 2
service.get(session, "https://api.test/x")
assert len(session.calls) == 2
def test_cache_max_age_never_reaches_the_session(self, service):
session = FakeSession(fresh_for(60))
service.get(session, "https://api.test/x", timeout=5, cache_max_age=30)
assert session.calls[0][1] == {"timeout": 5}
@pytest.mark.parametrize("status,headers", [
(200, {"Cache-Control": "no-store, max-age=60"}),
(200, {"Cache-Control": "no-cache, max-age=60"}),
(200, {"Cache-Control": "private, max-age=60"}),
(200, {"Cache-Control": "max-age=60", "Vary": "*"}),
(200, {"Cache-Control": "max-age=60", "Set-Cookie": "sid=1"}),
(200, {"Cache-Control": "max-age=0"}),
(200, {"Cache-Control": "max-age=soon"}),
(200, {"Cache-Control": "max-age=60", "Age": "60"}),
(200, {}),
(404, {"Cache-Control": "max-age=60"}),
(500, {"Cache-Control": "max-age=60"}),
])
def test_responses_that_must_not_be_reused_are_not(self, service, status, headers):
session = FakeSession(lambda url, kw: make_response(status, b"{}", headers=headers, url=url))
service.get(session, "https://api.test/x")
service.get(session, "https://api.test/x")
assert len(session.calls) == 2
assert service.snapshot()["response_cache"]["entries"] == 0
def test_a_stream_is_never_kept(self, service):
session = FakeSession(fresh_for(60))
service.get(session, "https://api.test/x", stream=True)
service.get(session, "https://api.test/x", stream=True)
assert len(session.calls) == 2
@pytest.mark.parametrize("second", [
{"params": {"d": 2}},
{"headers": {"Accept": "text/html"}},
{"headers": {"If-None-Match": '"x"'}}, # the caller's own revalidation
])
def test_requests_that_could_answer_differently_do_not_share(self, service, second):
session = FakeSession(fresh_for(60))
service.get(session, "https://api.test/x", params={"d": 1})
service.get(session, "https://api.test/x", **{"params": {"d": 1}, **second})
assert len(session.calls) == 2
def test_another_timeout_or_retry_policy_still_shares(self, service):
# A finished 200 is the same answer however long the caller would have
# waited or however often retried; only in-flight merging needs those.
plain, retrying = FakeSession(fresh_for(60)), FakeSession(fresh_for(60))
retrying.mount("https://", requests.adapters.HTTPAdapter(max_retries=Retry(total=5)))
service.get(plain, "https://api.test/x", timeout=5)
service.get(retrying, "https://api.test/x", timeout=30)
assert len(plain.calls) == 1 and retrying.calls == []
def test_a_session_with_cookies_only_reuses_its_own(self, service):
cookied, plain = FakeSession(fresh_for(60)), FakeSession(fresh_for(60))
cookied.cookies.set("sid", "secret")
service.get(cookied, "https://api.test/x")
service.get(plain, "https://api.test/x")
service.get(cookied, "https://api.test/x")
assert len(cookied.calls) == len(plain.calls) == 1
def test_size_bounds(self, clock):
svc = FetchService({"rate_limits": {}, "response_cache": {
"max_entry_bytes": 10, "max_bytes": 25, "max_entries": 10}},
clock=clock.now, sleep=clock.sleep)
big = FakeSession(fresh_for(60, body=b"x" * 11))
svc.get(big, "https://api.test/big")
assert svc.snapshot()["response_cache"]["entries"] == 0
small = FakeSession(fresh_for(60, body=b"y" * 10))
for name in "abc":
svc.get(small, f"https://api.test/{name}")
assert svc.snapshot()["response_cache"] == {"entries": 2, "bytes": 20}
svc.get(small, "https://api.test/a") # evicted, least recently used
assert len(small.calls) == 4
def test_expired_entries_are_dropped_on_insert(self, service, clock):
session = FakeSession(fresh_for(5))
service.get(session, "https://api.test/a")
clock.advance(6)
service.get(session, "https://api.test/b")
assert service.snapshot()["response_cache"]["entries"] == 1
def test_off_switch(self, clock):
svc = FetchService({"rate_limits": {}, "response_cache": {"enabled": False}},
clock=clock.now, sleep=clock.sleep)
session = FakeSession(fresh_for(60))
svc.get(session, "https://api.test/x")
svc.get(session, "https://api.test/x")
assert len(session.calls) == 2
assert svc.describe_config()["response_cache"] is False
def test_merged_callers_and_the_cache_together(self, service):
gate = threading.Event()
session = FakeSession(fresh_for(60), gate=gate)
a = threading.Thread(target=lambda: service.get(session, "https://api.test/x"))
a.start()
assert session.started.wait(5)
b = threading.Thread(target=lambda: service.get(session, "https://api.test/x"))
b.start()
_wait_for_waiters(service, 1)
gate.set()
a.join(5)
b.join(5)
service.get(session, "https://api.test/x")
assert len(session.calls) == 1
totals = _counters(service)
assert (totals["requests"], totals["merged"], totals["memo_hits"]) == (1, 1, 1)
def test_hits_are_counted_per_plugin_and_host(self, service):
session = FakeSession(fresh_for(60))
with plugin_scope("odds-ticker"):
service.get(session, "https://site.api.espn.com/x")
with plugin_scope("football-scoreboard"):
service.get(session, "https://site.api.espn.com/x")
snap = service.snapshot()
assert snap["plugins"]["football-scoreboard"]["memo_hits"] == 1
assert snap["plugins"]["football-scoreboard"]["requests"] == 0
assert snap["plugins"]["football-scoreboard"]["hosts"] == {"site.api.espn.com": 1}
assert snap["hosts"]["site.api.espn.com"]["memo_hits"] == 1
def test_the_odds_manager_never_takes_odds_older_than_its_interval(self, global_service, clock):
from unittest.mock import MagicMock
from src.base_odds_manager import BaseOddsManager
cache = MagicMock()
cache.get_with_auto_strategy.return_value = None
manager = BaseOddsManager(cache)
manager.session = FakeSession(fresh_for(450, body=b'{"count": 0, "items": []}'))
manager.get_odds("football", "nfl", "401", update_interval_seconds=60)
clock.advance(59)
manager.get_odds("football", "nfl", "401", update_interval_seconds=60)
assert len(manager.session.calls) == 1
clock.advance(2)
manager.get_odds("football", "nfl", "401", update_interval_seconds=60)
assert len(manager.session.calls) == 2
def test_the_api_helper_bounds_a_cached_get_by_its_ttl(self, global_service, clock):
from src.common.api_helper import APIHelper
helper = APIHelper()
helper.set_rate_limit(0)
helper.session = FakeSession(fresh_for(450, body=b'{"a": 1}'))
helper.get("https://api.test/x")
clock.advance(31)
helper.get("https://api.test/x") # no TTL: the 30 s default
assert len(helper.session.calls) == 2
clock.advance(31)
helper.cache_manager = NeverHits()
helper.get("https://api.test/x", cache_key="k", cache_ttl=60) # 31 s <= 60
assert len(helper.session.calls) == 2
class TestCacheHitCounters:
def test_a_shared_cache_hit(self, service):
with plugin_scope("odds-ticker"):
service.note_cache_hit("https://site.api.espn.com/")
counters = _counters(service, plugin="odds-ticker")
assert (counters["cache_hits"], counters["legacy_cache_hits"]) == (1, 0)
assert counters["hosts"] == {"site.api.espn.com": 1}
def test_a_legacy_hit(self, service):
service.note_cache_hit("https://site.api.espn.com/", legacy=True)
totals = _counters(service)
assert (totals["cache_hits"], totals["legacy_cache_hits"]) == (1, 1)
def test_a_read_that_avoided_no_request(self, service):
service.note_cache_hit("https://site.api.espn.com/", legacy=True, avoided_request=False)
totals = _counters(service)
assert (totals["cache_hits"], totals["legacy_cache_hits"]) == (0, 1)
def test_every_counter_is_in_every_snapshot(self, service):
snap = service.snapshot()
for name in ("memo_hits", "cache_hits", "legacy_cache_hits"):
assert snap["totals"][name] == 0
assert snap["response_cache"] == {"entries": 0, "bytes": 0}
+14 -4
View File
@@ -675,30 +675,40 @@ class TestTheControllersOwnScreens:
_push(dm, (255, 0, 0)) # the ticker's last frame
flags = _record_scrolling(dm, monkeypatch)
c = _core_screen_controller(dm)
DisplayController._blank_while_scheduled_off(c)
DisplayController._blank_while_scheduled_off(c, 60.0)
assert dm._presented[-1].getpixel((10, 30)) == (0, 0, 0)
assert flags == [False] # a static frame, not a freeze
assert not dm.is_currently_scrolling()
c._sleep_with_plugin_updates.assert_called_once_with(60)
def test_the_wifi_notice_shows_none_of_the_ticker(self, dm, monkeypatch):
from src.display_arbiter import WifiNotice
from src.display_controller import DisplayController
dm._scan_lag_bands = [(24, 48, 1)]
dm.set_scrolling_state(True, 1)
_push(dm, (255, 0, 0))
flags = _record_scrolling(dm, monkeypatch)
c = _core_screen_controller(dm)
c._check_wifi_status_message.return_value = {"message": "x", "expires_at": 1e12}
c._display_wifi_status_message.side_effect = lambda _s: _push(dm, (0, 0, 255)) and True
assert DisplayController._show_wifi_notice(c) is True
notice = WifiNotice(message="x", expires_at=1e12)
assert DisplayController._show_wifi_notice(c, notice, 0.5) is True
assert dm._presented[-1].getpixel((10, 30)) == (0, 0, 255)
assert flags == [False]
def test_no_notice_leaves_the_scroll_alone(self, dm):
"""With no notice the Arbiter does not pick the WiFi screen, so
nothing ends the scroll."""
from src.display_arbiter import Arbiter, ArbiterState, Source
from src.display_controller import DisplayController
dm.set_scrolling_state(True, 2)
c = _core_screen_controller(dm)
c.is_display_active = True
c.on_demand_schedule_override = False
c.sync_manager.is_follower_active.return_value = False
c._read_wifi_notice = types.MethodType(DisplayController._read_wifi_notice, c)
c._check_wifi_status_message.return_value = None
assert DisplayController._show_wifi_notice(c) is False
inputs = DisplayController._arbiter_inputs(c)
assert inputs.wifi_notice is None
assert Arbiter.decide(ArbiterState(), inputs, 0.0).source is Source.LEGACY
assert dm.is_currently_scrolling()
assert dm._frame_hold == 2
+101
View File
@@ -11,7 +11,9 @@ The cross-process tests use two CacheManagers over one temporary directory,
the arrangement of the real services (which share /var/cache/ledmatrix).
"""
import json
import os
import sys
import time
from pathlib import Path
from unittest.mock import MagicMock
@@ -19,6 +21,7 @@ import pytest
sys.path.insert(0, str(Path(__file__).parent.parent))
from src import display_watchdog # noqa: E402
from src.cache_manager import CacheManager # noqa: E402
from src.plugin_system import plugin_runtime as rt # noqa: E402
from src.plugin_system.plugin_manager import PluginManager # noqa: E402
@@ -305,6 +308,74 @@ class TestReader:
eager = _snapshot(self.NOW - 20, stale_after=0)
assert view_from_snapshot(eager, now=self.NOW).status == "live"
# -- render-loop liveness (the heartbeat /api/v3/health reads) --
MONO = 50_000.0
def _beat(self, age, pid=4242):
return {"pid": pid, "mono": self.MONO - age, "wall": self.NOW - age}
def test_a_stale_heartbeat_with_a_live_snapshot_is_stalled(self):
"""The publisher thread keeps writing while the render loop is
hung; the heartbeat says so, and the runtime status must agree with
/api/v3/health's display_loop: stalled."""
snapshot = _snapshot(self.NOW - 5, plugins={"clock": self.CLOCK}, pid=4242)
view = view_from_snapshot(
snapshot, now=self.NOW, now_mono=self.MONO,
heartbeat=self._beat(display_watchdog.HEARTBEAT_STALE_SECONDS + 1))
assert view.status == "stalled"
assert view.plugin("clock")["loaded"] is None # no frozen truth passed on
described = view.describe()
assert described["status"] == "stalled"
assert described["heartbeat_age_seconds"] == display_watchdog.HEARTBEAT_STALE_SECONDS + 1
def test_the_threshold_is_the_health_checks(self):
snapshot = _snapshot(self.NOW - 5, pid=4242)
limit = display_watchdog.HEARTBEAT_STALE_SECONDS
fresh = view_from_snapshot(snapshot, now=self.NOW, now_mono=self.MONO,
heartbeat=self._beat(limit - 0.5))
assert fresh.status == "live"
assert fresh.describe()["heartbeat_age_seconds"] == limit - 0.5
assert view_from_snapshot(snapshot, now=self.NOW, now_mono=self.MONO,
heartbeat=self._beat(limit)).status == "stalled"
@pytest.mark.parametrize("heartbeat", [
None, # dev server, Windows, starting up
{"pid": 9999, "mono": 0.0}, # another process: a restarted display
{"pid": "4242", "mono": 0.0}, # unparseable pid
{"pid": 4242}, # no time in it
])
def test_a_heartbeat_that_says_nothing_leaves_it_live(self, heartbeat):
snapshot = _snapshot(self.NOW - 5, plugins={"clock": self.CLOCK}, pid=4242)
view = view_from_snapshot(snapshot, now=self.NOW, now_mono=self.MONO,
heartbeat=heartbeat)
assert view.status == "live"
assert view.plugin("clock")["loaded"] is True
def test_a_stale_snapshot_stays_stale_whatever_the_heartbeat(self):
snapshot = _snapshot(self.NOW - 181, pid=4242)
assert view_from_snapshot(snapshot, now=self.NOW, now_mono=self.MONO,
heartbeat=self._beat(500)).status == "stale"
def test_a_snapshot_from_a_dead_process_is_stale_at_once(self):
"""After a watchdog kill systemd removes the heartbeat's directory, so
nothing goes stale; the publisher's pid being gone is the signal."""
snapshot = _snapshot(self.NOW - 5, plugins={"clock": self.CLOCK}, pid=4242)
dead = view_from_snapshot(snapshot, now=self.NOW,
process_alive=lambda pid: False)
assert dead.status == "stale"
assert dead.plugin("clock")["loaded"] is None
for answer in (True, None): # alive, or this platform cannot tell
assert view_from_snapshot(snapshot, now=self.NOW,
process_alive=lambda pid, a=answer: a).status == "live"
def test_process_exists(self):
if os.name != "posix":
assert rt.process_exists(os.getpid()) is None
else:
assert rt.process_exists(os.getpid()) is True
assert rt.process_exists(0) is None
def test_no_cache_manager_or_a_failing_one_is_unknown(self):
assert read_plugin_runtime(None).status == "unknown"
cache = MagicMock()
@@ -424,6 +495,36 @@ class TestInstalledPluginsRoute:
assert runtime["age_seconds"] > rt.STALE_AFTER
assert plugins["clock"]["loaded"] is None
def _write_heartbeat(self, tmp_path, monkeypatch, age):
path = tmp_path / "display-heartbeat.json"
path.write_text(json.dumps({"pid": os.getpid(), "mono": time.monotonic() - age,
"wall": time.time() - age}), encoding="utf-8")
monkeypatch.setattr(display_watchdog, "HEARTBEAT_PATH", str(path))
def test_hung_render_loop_is_stalled_not_live(self, web_listing, tmp_path, monkeypatch):
"""The publisher's own thread still ticks while the render loop is
stuck; the heartbeat it shares a process with has gone stale."""
display_cache, get = web_listing
self._display(display_cache).tick() # a fresh snapshot, this pid
self._write_heartbeat(tmp_path, monkeypatch,
display_watchdog.HEARTBEAT_STALE_SECONDS + 30)
plugins, runtime = get()
assert runtime["status"] == "stalled"
assert runtime["heartbeat_age_seconds"] >= display_watchdog.HEARTBEAT_STALE_SECONDS
assert plugins["clock"]["loaded"] is None and plugins["clock"]["state"] is None
def test_fresh_heartbeat_keeps_it_live(self, web_listing, tmp_path, monkeypatch):
display_cache, get = web_listing
self._display(display_cache).tick()
self._write_heartbeat(tmp_path, monkeypatch, 1)
plugins, runtime = get()
assert runtime["status"] == "live"
assert plugins["clock"]["loaded"] is True
class TestDisplayControllerStopsThePublisher:
def test_cleanup_publishes_stopped(self, monkeypatch):
+67 -4
View File
@@ -41,7 +41,9 @@ def _url(path):
def test_the_module_directories_hold_modules():
assert {p.name for p in MODULES} >= {"boot.js", "registry.js", "api.js", "facade.js", "cache.js"}
assert {p.name for p in MODULES} >= {"boot.js", "registry.js", "api.js", "facade.js", "cache.js",
"durations.js", "operation-history.js", "raw-json.js",
"backup-restore.js"}
for directory in MODULE_DIRS:
# node needs this to import them in the JS tests; browsers ignore it.
assert '"type": "module"' in (directory / "package.json").read_text(encoding="utf-8")
@@ -94,10 +96,71 @@ def test_every_import_resolves_inside_the_module_tree(module):
f"{module.name}: {spec!r} leaves js/core and js/pages")
# Converted so far: page name -> (its partial, the route that serves it). The
# migration order is in docs/WEB_FRONTEND_ARCHITECTURE.md.
CONVERTED = {
"cache": ("cache.html", "/partials/cache"),
"durations": ("durations.html", "/partials/durations"),
"operation-history": ("operation_history.html", "/partials/operation-history"),
"raw-json": ("raw_json.html", "/partials/raw-json"),
"backup-restore": ("backup_restore.html", "/partials/backup-restore"),
}
# Old window.* names that moved into a page module. Each stays as a
# deprecated alias in boot.js that forwards to the module's export.
ALIASES = {
"cache": ["deleteCacheFile"],
"raw-json": ["formatJson", "manualValidateJson", "validateJSON",
"saveMainConfig", "saveSecretsConfig"],
"backup-restore": ["exportBackup", "loadBackupList", "validateRestoreFile",
"clearRestore", "runRestore"],
}
def _boot():
return (JS / "core" / "boot.js").read_text(encoding="utf-8")
def _registered_pages():
boot = (JS / "core" / "boot.js").read_text(encoding="utf-8")
return re.findall(r"registry\.register\('([\w-]+)',\s*function\(\)\s*\{\s*return import\('\.\./pages/([\w-]+)\.js'\)",
boot)
return re.findall(r"'([\w-]+)':\s*page\(function\(\)\s*\{\s*return import\('\.\./pages/([\w-]+)\.js'\)",
_boot())
def test_every_converted_page_is_registered():
assert dict(_registered_pages()) == {name: name for name in CONVERTED}
@pytest.mark.parametrize("name", sorted(CONVERTED))
def test_a_converted_partial_roots_its_page(client, name):
partial, route = CONVERTED[name]
text = (PARTIALS / partial).read_text(encoding="utf-8")
assert f'data-page="{name}"' in text
assert "<script" not in text.lower()
# Its buttons are wired by the module, not by inline handlers naming
# globals (which would reach the code only through a deprecated alias).
assert "onclick=" not in text.lower()
module = (JS / "pages" / f"{name}.js").read_text(encoding="utf-8")
assert re.search(r"^export function init\(root, ctx\)", module, re.M)
# Rendered by the real route, the root is there exactly once.
resp = client.get(route)
assert resp.status_code == 200, route
assert resp.get_data(as_text=True).count(f'data-page="{name}"') == 1, route
@pytest.mark.parametrize("name", sorted(ALIASES))
def test_moved_globals_stay_as_aliases(name):
boot = _boot()
module = (JS / "pages" / f"{name}.js").read_text(encoding="utf-8")
for global_name in ALIASES[name]:
assert f"'{global_name}'" in boot, f"boot.js does not alias {global_name}"
assert re.search(rf"^export (?:function|const) {global_name}\b", module, re.M), (
f"pages/{name}.js does not export {global_name}")
# No template defines them any more.
for partial in PARTIALS.glob("*.html"):
text = partial.read_text(encoding="utf-8")
for global_name in ALIASES[name]:
assert f"window.{global_name} =" not in text, partial.name
assert f"function {global_name}(" not in text, partial.name
def test_every_registered_page_has_its_module_and_partial():
+2 -1
View File
@@ -52,7 +52,8 @@ def get_installed_plugins():
display publishes (src/plugin_system/plugin_runtime.py), and only while
that snapshot is live: when the display is stopped, hung or has never
published, they are null and ``data.runtime.status`` says why
(``stale``, ``stopped``, ``unknown``) instead of passing on old truth.
(``stalled``, ``stale``, ``stopped``, ``unknown``) instead of passing on
old truth.
Health, metrics and errors are served by /plugins/health,
/plugins/metrics and /errors.
"""
+5
View File
@@ -4,6 +4,7 @@
* const api = createApi();
* const body = await api.get('/api/v3/cache/list', { signal });
* await api.post('/api/v3/cache/delete', { key }, { signal });
* await api.request('POST', '/api/v3/backup/validate', { body: formData });
*
* Every call resolves to the parsed JSON body, or rejects with an ApiError:
* error.status the HTTP status (0 when no HTTP answer arrived)
@@ -77,6 +78,10 @@ export function createApi(options = {}) {
if (opts.json !== undefined) {
init.headers['Content-Type'] = 'application/json';
init.body = JSON.stringify(opts.json);
} else if (opts.body !== undefined) {
// Sent as it is (a FormData upload, say); the browser sets the
// Content-Type, multipart boundary included.
init.body = opts.body;
}
let response;
+40 -4
View File
@@ -21,13 +21,49 @@ const registry = createRegistry({
});
const facade = installFacade(window, createFacade(window, api, registry));
/**
* A page's loader. It remembers the module once loaded, so an alias of a
* synchronous old global (validateJSON returns a boolean) can answer
* synchronously while its page is on screen.
*/
function page(load) {
let module = null;
const loader = function() {
return Promise.resolve(load()).then(function(loaded) { module = loaded; return loaded; });
};
loader.loaded = function() { return module; };
return loader;
}
// Converted pages. Each loads on first use: its module is fetched only when
// its partial (data-page="<name>") first appears.
registry.register('cache', function() { return import('../pages/cache.js'); });
const pages = {
'cache': page(function() { return import('../pages/cache.js'); }),
'durations': page(function() { return import('../pages/durations.js'); }),
'operation-history': page(function() { return import('../pages/operation-history.js'); }),
'raw-json': page(function() { return import('../pages/raw-json.js'); }),
'backup-restore': page(function() { return import('../pages/backup-restore.js'); }),
};
Object.keys(pages).forEach(function(name) { registry.register(name, pages[name]); });
/** Keep window[name] working: forward to the page module's export of the same name. */
function alias(pageName, name, replacement) {
const loader = pages[pageName];
facade.deprecate(name, function() {
const args = arguments;
const module = loader.loaded();
if (module) return module[name].apply(null, args);
return loader().then(function(loaded) { return loaded[name].apply(null, args); });
}, replacement);
}
// Old globals the converted pages used to define.
facade.deprecate('deleteCacheFile', function(key) {
return import('../pages/cache.js').then(function(page) { return page.deleteCacheFile(key); });
}, "the Cache tab's Delete buttons");
alias('cache', 'deleteCacheFile', "the Cache tab's Delete buttons");
['formatJson', 'manualValidateJson', 'validateJSON', 'saveMainConfig', 'saveSecretsConfig'].forEach(function(name) {
alias('raw-json', name, "the Config Editor tab's buttons");
});
['exportBackup', 'loadBackupList', 'validateRestoreFile', 'clearRestore', 'runRestore'].forEach(function(name) {
alias('backup-restore', name, "the Backup & Restore tab's buttons");
});
registry.start();
@@ -0,0 +1,373 @@
/*
* pages/backup-restore.js -- the Backup & Restore tab
* (templates/v3/partials/backup_restore.html).
*
* The buttons carry data-action; one delegated listener on the page root
* handles them all, including each history row's Delete. Everything the
* server sends back (file names, host names, plugin ids, error messages) is
* drawn with textContent.
*
* Reads (the summary, the history list, inspecting a file) carry ctx.signal
* and are cancelled when the page is swapped away. Writes (export, delete,
* restore) are not: the server always finishes them and the result is still
* reported in a notification, but nothing is drawn into a page that is gone.
*
* The old globals (exportBackup, loadBackupList, validateRestoreFile,
* clearRestore, runRestore) are deprecated aliases made in core/boot.js; they
* forward to the exports at the bottom of this file.
*/
const URLS = {
preview: '/api/v3/backup/preview',
list: '/api/v3/backup/list',
exportZip: '/api/v3/backup/export',
validate: '/api/v3/backup/validate',
restore: '/api/v3/backup/restore',
download: '/api/v3/backup/download/',
remove: '/api/v3/backup/',
};
// Checkbox id -> RestoreOptions key (backup.py's _RESTORE_OPTION_KEYS).
const RESTORE_OPTIONS = [
['opt-config', 'restore_config'],
['opt-secrets', 'restore_secrets'],
['opt-wifi', 'restore_wifi'],
['opt-fonts', 'restore_fonts'],
['opt-plugin-uploads', 'restore_plugin_uploads'],
['opt-reinstall', 'reinstall_plugins'],
];
// The mounted page, for the deprecated aliases (boot.js).
let active = null;
function isQuiet(error) {
return !!error && (error.name === 'AbortError' || error.loginRequired);
}
function message(error) {
return (error && error.message) || 'Unknown error';
}
export function formatSize(bytes) {
if (!bytes) return '0 B';
const units = ['B', 'KB', 'MB', 'GB'];
let i = 0;
let size = bytes;
while (size >= 1024 && i < units.length - 1) { size /= 1024; i++; }
return size.toFixed(i === 0 ? 0 : 1) + ' ' + units[i];
}
function el(doc, tag, className, text) {
const node = doc.createElement(tag);
if (className) node.className = className;
if (text !== undefined) node.textContent = text;
return node;
}
/** "<strong>label</strong> value" as one line. */
function labelled(doc, tag, label, value) {
const line = el(doc, tag);
line.append(el(doc, 'strong', null, label), doc.createTextNode(' ' + value));
return line;
}
function list(values, map, separator) {
return (values || []).map(map || String).join(separator || ', ');
}
function setBusy(button, busy, idleIcon, idleText, busyText) {
const doc = button.ownerDocument;
button.disabled = busy;
button.replaceChildren(el(doc, 'i', busy ? 'fas fa-spinner fa-spin mr-2' : 'fas ' + idleIcon + ' mr-2'),
doc.createTextNode(busy ? busyText : idleText));
}
function elements(root) {
const $ = function(id) { return root.querySelector('#' + id); };
return {
preview: $('export-preview'),
exportButton: $('export-backup-btn'),
fileInput: $('restore-file-input'),
restorePreview: $('restore-preview'),
restorePreviewBody: $('restore-preview-body'),
restoreButton: $('run-restore-btn'),
result: $('restore-result'),
history: $('backup-history'),
};
}
/** The "what a new backup would hold" summary on the Export card. */
export function loadPreview(ctx) {
const box = ctx.state.els.preview;
const doc = ctx.root.ownerDocument;
return ctx.api.get(URLS.preview, { signal: ctx.signal }).then(function(body) {
const d = body.data || {};
const fonts = Array.isArray(d.user_fonts) ? d.user_fonts : [];
const ul = el(doc, 'ul', 'list-disc pl-5 space-y-1');
function item(label, value, after) {
const li = el(doc, 'li', null, label + ': ');
li.append(el(doc, 'strong', null, String(value)));
if (after) li.append(doc.createTextNode(' ' + after));
ul.append(li);
}
item('Main config', d.has_config ? 'yes' : 'no');
item('Secrets', d.has_secrets ? 'yes' : 'no');
item('WiFi config', d.has_wifi ? 'yes' : 'no');
item('User fonts', fonts.length, fonts.length ? '(' + list(fonts) + ')' : '');
item('Plugin image uploads', d.plugin_uploads || 0, 'file(s)');
item('Installed plugins', (d.plugins || []).length);
box.replaceChildren(ul);
}).catch(function(err) {
if (isQuiet(err)) return;
box.textContent = 'Could not load preview: ' + message(err);
});
}
function historyTable(doc, entries) {
const table = el(doc, 'table', 'min-w-full divide-y divide-gray-200');
const head = el(doc, 'tr');
['Filename', 'Size', 'Created'].forEach(function(label) {
head.append(el(doc, 'th', 'text-left py-2', label));
});
head.append(el(doc, 'th'));
const thead = el(doc, 'thead');
thead.append(head);
const tbody = el(doc, 'tbody', 'divide-y divide-gray-100');
entries.forEach(function(entry) {
const name = String(entry.filename);
const row = el(doc, 'tr');
const actions = el(doc, 'td', 'py-2 text-right space-x-2');
const link = el(doc, 'a', 'text-blue-600 hover:underline', 'Download');
link.href = URLS.download + encodeURIComponent(name);
const remove = el(doc, 'button', 'text-red-600 hover:underline', 'Delete');
remove.type = 'button';
remove.dataset.action = 'delete';
remove.dataset.filename = name;
remove.setAttribute('aria-label', 'Delete ' + name);
actions.append(link, doc.createTextNode(' '), remove);
row.append(el(doc, 'td', 'py-2 font-mono text-xs', name),
el(doc, 'td', 'py-2', formatSize(entry.size)),
el(doc, 'td', 'py-2', String(entry.created_at)),
actions);
tbody.append(row);
});
table.append(thead, tbody);
return table;
}
/** The Backup history card. A newer load supersedes an older one. */
export function loadList(ctx) {
const box = ctx.state.els.history;
const doc = ctx.root.ownerDocument;
const seq = (ctx.state.listSeq || 0) + 1;
ctx.state.listSeq = seq;
box.textContent = 'Loading…';
return ctx.api.get(URLS.list, { signal: ctx.signal }).then(function(body) {
if (ctx.state.listSeq !== seq) return;
const entries = Array.isArray(body.data) ? body.data : [];
box.replaceChildren(entries.length ? historyTable(doc, entries)
: el(doc, 'p', null, 'No backups have been created yet.'));
}).catch(function(err) {
if (isQuiet(err) || ctx.state.listSeq !== seq) return;
box.textContent = 'Could not load backups: ' + message(err);
});
}
/** Create a backup, start its download, and refresh the history. */
export function exportZip(ctx) {
const button = ctx.state.els.exportButton;
if (button.disabled) return Promise.resolve(false);
setBusy(button, true, 'fa-download', 'Download backup', 'Creating…');
return ctx.api.post(URLS.exportZip, {}).then(function(body) {
ctx.notify('Backup created: ' + body.filename, 'success');
if (ctx.signal.aborted) return true;
ctx.root.ownerDocument.defaultView.location.assign(URLS.download + encodeURIComponent(body.filename));
return loadList(ctx).then(function() { return true; });
}).catch(function(err) {
if (!isQuiet(err)) ctx.notify('Export failed: ' + message(err), 'error');
return false;
}).finally(function() {
setBusy(button, false, 'fa-download', 'Download backup');
});
}
/** Ask, then delete one stored backup. */
export function deleteBackup(ctx, filename) {
const win = ctx.root.ownerDocument.defaultView;
if (!win.confirm('Delete ' + filename + '?')) return Promise.resolve(false);
return ctx.api.del(URLS.remove + encodeURIComponent(filename)).then(function() {
ctx.notify('Backup deleted', 'success');
if (!ctx.signal.aborted) loadList(ctx);
return true;
}).catch(function(err) {
if (!isQuiet(err)) ctx.notify('Delete failed: ' + message(err), 'error');
return false;
});
}
function hideRestore(ctx) {
ctx.state.inspected = null;
ctx.state.els.restorePreview.classList.add('hidden');
ctx.state.els.result.classList.add('hidden');
}
function renderManifest(ctx, manifest) {
const doc = ctx.root.ownerDocument;
const detected = manifest.detected_contents || [];
const plugins = manifest.plugins || [];
ctx.state.els.restorePreviewBody.replaceChildren(
labelled(doc, 'div', 'Created:', String(manifest.created_at || 'unknown')),
labelled(doc, 'div', 'Source host:', String(manifest.hostname || 'unknown')),
labelled(doc, 'div', 'LEDMatrix version:', String(manifest.ledmatrix_version || 'unknown')),
labelled(doc, 'div', 'Includes:', detected.length ? list(detected) : '(nothing detected)'),
labelled(doc, 'div', 'Plugins referenced:',
plugins.length ? list(plugins, function(p) { return String(p && p.plugin_id); }) : 'none'),
);
ctx.state.els.restorePreview.classList.remove('hidden');
}
/** Upload the chosen file for inspection and show what it holds. */
export function inspect(ctx) {
const input = ctx.state.els.fileInput;
const file = input.files && input.files[0];
if (!file) {
ctx.notify('Choose a backup file first', 'error');
return Promise.resolve(false);
}
const win = ctx.root.ownerDocument.defaultView;
const form = new win.FormData();
form.append('backup_file', file);
return ctx.api.request('POST', URLS.validate, { body: form, signal: ctx.signal }).then(function(body) {
// A different file picked while this one was being inspected.
if (!input.files || input.files[0] !== file) return false;
ctx.state.inspected = file;
renderManifest(ctx, body.data || {});
return true;
}).catch(function(err) {
if (!isQuiet(err)) ctx.notify('Invalid backup: ' + message(err), 'error');
return false;
});
}
/** The Cancel button: forget the inspected file. */
export function clear(ctx) {
hideRestore(ctx);
ctx.state.els.fileInput.value = '';
}
function renderResult(ctx, data, partial) {
const doc = ctx.root.ownerDocument;
const result = ctx.state.els.result;
result.className = (partial
? 'bg-yellow-50 border-yellow-200 text-yellow-800'
: 'bg-green-50 border-green-200 text-green-800') + ' border rounded-md p-4';
const lines = [
el(doc, 'h3', 'font-medium mb-2', partial ? 'Restore complete with warnings' : 'Restore complete'),
labelled(doc, 'div', 'Restored:', list(data.restored) || 'none'),
labelled(doc, 'div', 'Skipped:', list(data.skipped) || 'none'),
labelled(doc, 'div', 'Plugins installed:', list(data.plugins_installed) || 'none'),
labelled(doc, 'div', 'Plugins failed:', list(data.plugins_failed, function(p) {
return String(p.plugin_id) + ' (' + String(p.error) + ')';
}) || 'none'),
labelled(doc, 'div', 'Errors:', list(data.errors, String, '; ') || 'none'),
];
if ((data.restored || []).length || (data.plugins_installed || []).length) {
lines.push(el(doc, 'p', 'mt-2', 'Restart the display service to apply all changes.'));
}
result.replaceChildren(...lines);
}
/** Ask, then restore the inspected file with the chosen options. */
export function restore(ctx) {
const file = ctx.state.inspected;
if (!file) {
ctx.notify('Inspect the file before restoring', 'error');
return Promise.resolve(false);
}
const button = ctx.state.els.restoreButton;
if (button.disabled) return Promise.resolve(false); // one already running
const win = ctx.root.ownerDocument.defaultView;
if (!win.confirm('Restore from this backup? Current configuration will be overwritten.')) {
return Promise.resolve(false);
}
const options = {};
RESTORE_OPTIONS.forEach(function(pair) {
const box = ctx.root.querySelector('#' + pair[0]);
options[pair[1]] = !!(box && box.checked);
});
const form = new win.FormData();
form.append('backup_file', file);
form.append('options', JSON.stringify(options));
setBusy(button, true, 'fa-upload', 'Restore now', 'Restoring…');
return ctx.api.request('POST', URLS.restore, { body: form }).then(function(body) {
const data = body.data || {};
const partial = (data.plugins_failed || []).length > 0 || (data.errors || []).length > 0;
if (!ctx.signal.aborted) {
renderResult(ctx, data, partial);
ctx.state.els.result.classList.remove('hidden');
}
ctx.notify(partial ? 'Restore complete with warnings' : 'Restore complete',
partial ? 'warning' : 'success');
return true;
}).catch(function(err) {
if (isQuiet(err)) return false;
const errors = (err.body && err.body.data && err.body.data.errors) || [];
ctx.notify('Restore failed: ' + ((err.body && err.body.message) || errors.join('; ')
|| err.message || 'Restore had errors'), 'error');
return false;
}).finally(function() {
setBusy(button, false, 'fa-upload', 'Restore now');
});
}
const ACTIONS = {
'export': exportZip,
'refresh': loadList,
'inspect': inspect,
'restore': restore,
'cancel': clear,
};
export function init(root, ctx) {
const els = elements(root);
ctx.state.els = els;
ctx.state.inspected = null;
const on = { signal: ctx.signal };
root.addEventListener('click', function(event) {
const button = event.target.closest('button[data-action]');
if (!button || !root.contains(button)) return;
if (button.dataset.action === 'delete') {
deleteBackup(ctx, button.dataset.filename);
return;
}
const action = ACTIONS[button.dataset.action];
if (action) action(ctx);
}, on);
// A newly picked file has to be inspected again before it can be restored.
els.fileInput.addEventListener('change', function() { hideRestore(ctx); }, on);
active = ctx;
loadPreview(ctx);
loadList(ctx);
}
export function destroy(root, ctx) {
if (active === ctx) active = null;
}
// ── the old globals, kept as deprecated aliases (boot.js) ────────────────────
function forward(action) {
return function() { return active ? action(active) : Promise.resolve(false); };
}
/** window.exportBackup() */
export const exportBackup = forward(exportZip);
/** window.loadBackupList() */
export const loadBackupList = forward(loadList);
/** window.validateRestoreFile() */
export const validateRestoreFile = forward(inspect);
/** window.clearRestore() */
export const clearRestore = forward(clear);
/** window.runRestore() */
export const runRestore = forward(restore);
@@ -0,0 +1,50 @@
/*
* pages/durations.js -- the Rotation & Durations tab
* (templates/v3/partials/durations.html).
*
* The form itself is plain htmx (hx-post to /api/v3/config/main), so the only
* code here starts the shared drag-and-drop rotation-order list
* (static/v3/js/widgets/plugin-order-list.js, the same widget the Vegas
* section of the Display tab uses). Its plugin-list request carries
* ctx.signal, so a swap cancels it.
*
* The widget is a classic deferred script and normally loads before this
* module runs; the short retry covers a page that arrives first anyway.
*/
const CONTAINER_ID = 'rotation_plugin_order';
const ORDER_INPUT_ID = 'rotation_plugin_order_value';
const RETRY_MS = 100;
const MAX_TRIES = 50;
export function init(root, ctx) {
const container = root.querySelector('#' + CONTAINER_ID);
if (!container) return;
const win = root.ownerDocument.defaultView;
let tries = 0;
function start() {
ctx.state.timer = null;
if (ctx.signal.aborted) return;
const widget = win.PluginOrderList;
if (!widget) {
if (++tries < MAX_TRIES) {
ctx.state.timer = win.setTimeout(start, RETRY_MS);
} else {
// Say so rather than showing "Loading..." for ever.
container.textContent = 'Could not load the reorder widget — reload the page to try again.';
container.className = 'text-sm text-red-500';
}
return;
}
widget.init({ containerId: CONTAINER_ID, orderInputId: ORDER_INPUT_ID, signal: ctx.signal });
}
start();
}
export function destroy(root, ctx) {
if (ctx.state.timer) {
root.ownerDocument.defaultView.clearTimeout(ctx.state.timer);
ctx.state.timer = null;
}
}
@@ -0,0 +1,283 @@
/*
* pages/operation-history.js -- the Operation History tab
* (templates/v3/partials/operation_history.html).
*
* A read-only audit list: the page loads up to 1,000 records once, then
* filters, searches and pages through them in the browser. Rows are built
* with textContent, never markup strings, so a plugin id or error message
* from the log is always shown as text.
*
* Every listener and read request carries ctx.signal. Clearing the history is
* a write, so it is left to finish if the page is swapped away mid-request;
* its result is then only reported, not drawn.
*/
const HISTORY_URL = '/api/v3/plugins/operation/history';
const LIST_URL = HISTORY_URL + '?limit=1000';
const PLUGINS_URL = '/api/v3/plugins/installed';
const PAGE_SIZE = 50;
const SEARCH_DELAY_MS = 300;
const STATUS_CLASSES = {
success: 'bg-green-100 text-green-800',
failed: 'bg-red-100 text-red-800',
};
const TYPE_CLASSES = {
install: 'bg-blue-100 text-blue-800',
update: 'bg-purple-100 text-purple-800',
uninstall: 'bg-red-100 text-red-800',
enable: 'bg-green-100 text-green-800',
disable: 'bg-yellow-100 text-yellow-800',
configure: 'bg-teal-100 text-teal-800',
};
const NEUTRAL_CLASS = 'bg-gray-100 text-gray-800';
function isQuiet(error) {
// A cancelled request (the page was swapped away) or the login redirect
// (the browser is already leaving): nothing to tell the user.
return !!error && (error.name === 'AbortError' || error.loginRequired);
}
/**
* The server says "completed" or "success" for a success and "error" or
* "failed" for a failure; the status filter's values are success/failed.
*/
export function normalizeStatus(status) {
const s = String(status || '').toLowerCase();
if (s === 'completed' || s === 'success') return 'success';
if (s === 'error' || s === 'failed') return 'failed';
return s;
}
/** The short "v1.2 | commit: abc1234" summary of a record's details. */
export function detailsSummary(details) {
if (!details || typeof details !== 'object') return '';
const parts = [];
if (details.version) parts.push('v' + details.version);
if (details.branch) parts.push('branch: ' + details.branch);
if (details.commit) parts.push('commit: ' + String(details.commit).substring(0, 7));
if (details.previous_commit && details.commit && details.previous_commit !== details.commit) {
parts.push('from: ' + String(details.previous_commit).substring(0, 7));
}
if (details.preserve_config !== undefined) {
parts.push(details.preserve_config ? 'config preserved' : 'config removed');
}
return parts.join(' | ');
}
/** The records that pass the current filters. */
export function filterRecords(records, filters) {
const search = (filters.search || '').toLowerCase();
return records.filter(function(record) {
if (filters.plugin && record.plugin_id !== filters.plugin) return false;
if (filters.type && record.operation_type !== filters.type) return false;
if (filters.status && normalizeStatus(record.status) !== filters.status) return false;
if (search) {
const searchable = [
record.operation_type, record.plugin_id, record.status, record.user, record.error,
JSON.stringify(record.details || {}),
].join(' ').toLowerCase();
if (!searchable.includes(search)) return false;
}
return true;
});
}
function elements(root) {
const $ = function(id) { return root.querySelector('#' + id); };
return {
plugin: $('history-plugin-filter'),
type: $('history-type-filter'),
status: $('history-status-filter'),
search: $('history-search'),
refresh: $('refresh-history-btn'),
clear: $('clear-history-btn'),
tbody: $('history-table-body'),
start: $('history-start'),
end: $('history-end'),
total: $('history-total'),
prev: $('history-prev-btn'),
next: $('history-next-btn'),
};
}
function el(doc, tag, className, text) {
const node = doc.createElement(tag);
if (className) node.className = className;
if (text !== undefined) node.textContent = text;
return node;
}
function messageRow(doc, text) {
const row = el(doc, 'tr');
const cell = el(doc, 'td', 'px-6 py-4 text-center text-gray-500', text);
cell.colSpan = 6;
row.append(cell);
return row;
}
function badge(doc, className, text) {
return el(doc, 'span', 'px-2 py-1 text-xs font-medium rounded ' + className, text);
}
function historyRow(doc, record) {
const row = el(doc, 'tr', 'hover:bg-gray-50');
const when = new Date(record.timestamp || record.created_at || Date.now());
const status = normalizeStatus(record.status);
const type = record.operation_type || 'unknown';
const typeCell = el(doc, 'td', 'px-6 py-4 whitespace-nowrap text-sm');
typeCell.append(badge(doc, TYPE_CLASSES[String(type).toLowerCase()] || NEUTRAL_CLASS, String(type)));
const statusCell = el(doc, 'td', 'px-6 py-4 whitespace-nowrap text-sm');
statusCell.append(badge(doc, STATUS_CLASSES[status] || NEUTRAL_CLASS, status));
const detailsCell = el(doc, 'td', 'px-6 py-4 text-sm text-gray-500');
const summary = detailsSummary(record.details);
detailsCell.append(summary ? el(doc, 'span', 'text-xs text-gray-500', summary) : doc.createTextNode('-'));
if (record.error) detailsCell.append(el(doc, 'div', 'text-red-600 text-xs mt-1', String(record.error)));
row.append(
el(doc, 'td', 'px-6 py-4 whitespace-nowrap text-sm text-gray-900', when.toLocaleString()),
typeCell,
el(doc, 'td', 'px-6 py-4 whitespace-nowrap text-sm text-gray-900', String(record.plugin_id || '-')),
statusCell,
el(doc, 'td', 'px-6 py-4 whitespace-nowrap text-sm text-gray-500', String(record.user || '-')),
detailsCell,
);
return row;
}
function render(ctx) {
const s = ctx.state;
const { tbody, start: startEl, end: endEl, total, prev, next } = s.els;
const doc = ctx.root.ownerDocument;
const pages = Math.max(1, Math.ceil(s.filtered.length / PAGE_SIZE));
if (s.page > pages) s.page = pages;
const start = (s.page - 1) * PAGE_SIZE;
const end = Math.min(start + PAGE_SIZE, s.filtered.length);
const rows = s.filtered.slice(start, end);
tbody.replaceChildren(...(rows.length
? rows.map(function(record) { return historyRow(doc, record); })
: [messageRow(doc, 'No operations found')]));
// Kept in step even when nothing matches, so "Showing 1 to 50" does not
// linger under an empty table.
startEl.textContent = String(s.filtered.length ? start + 1 : 0);
endEl.textContent = String(end);
total.textContent = String(s.filtered.length);
prev.disabled = s.page <= 1;
next.disabled = end >= s.filtered.length;
}
/** Re-apply the filters to the loaded records and draw the first page. */
export function applyFilters(ctx, keepPage) {
const { plugin, type, status, search } = ctx.state.els;
ctx.state.filtered = filterRecords(ctx.state.records, {
plugin: plugin.value, type: type.value, status: status.value, search: search.value,
});
if (!keepPage) ctx.state.page = 1;
render(ctx);
}
/** Fetch the history and draw it. A newer load supersedes an older one. */
export function load(ctx) {
const seq = (ctx.state.seq || 0) + 1;
ctx.state.seq = seq;
return ctx.api.get(LIST_URL, { signal: ctx.signal }).then(function(body) {
if (ctx.state.seq !== seq) return;
ctx.state.records = Array.isArray(body.data) ? body.data : [];
applyFilters(ctx, true);
}).catch(function(err) {
if (isQuiet(err) || ctx.state.seq !== seq) return;
ctx.notify(err.network || !err.status
? 'Error loading operation history'
: 'Failed to load operation history', 'error');
});
}
function installedPlugins(ctx) {
// PluginAPI (js/plugins/api_client.js) caches the installed list it
// shares with the Plugin Manager, so prefer it: a reloaded tab then costs
// no extra request. Without it, ask the API directly.
const pluginApi = ctx.root.ownerDocument.defaultView.PluginAPI;
if (pluginApi && typeof pluginApi.getInstalledPlugins === 'function') {
return Promise.resolve(pluginApi.getInstalledPlugins());
}
return ctx.api.get(PLUGINS_URL, { signal: ctx.signal }).then(function(body) {
return (body.data && Array.isArray(body.data.plugins)) ? body.data.plugins : [];
});
}
/** Fill the plugin filter with the installed plugins' ids. */
export function loadPluginFilter(ctx) {
return installedPlugins(ctx).then(function(plugins) {
if (ctx.signal.aborted || !Array.isArray(plugins)) return;
const doc = ctx.root.ownerDocument;
const select = ctx.state.els.plugin;
plugins.map(function(p) { return p && p.id; }).filter(Boolean).forEach(function(id) {
const option = doc.createElement('option');
option.value = String(id);
option.textContent = String(id);
select.append(option);
});
}).catch(function(err) {
if (!isQuiet(err)) ctx.root.ownerDocument.defaultView.console.error('Error loading plugins for filter:', err);
});
}
/** Ask, then clear the whole history. */
export function clearHistory(ctx) {
const win = ctx.root.ownerDocument.defaultView;
if (!win.confirm('Are you sure you want to clear the operation history? This cannot be undone.')) {
return Promise.resolve(false);
}
return ctx.api.del(HISTORY_URL).then(function() {
ctx.state.seq = (ctx.state.seq || 0) + 1; // an older load must not redraw the old list
if (ctx.signal.aborted) return true;
ctx.state.records = [];
applyFilters(ctx);
return true;
}).catch(function(err) {
if (isQuiet(err)) return false;
ctx.notify(err.network || !err.status ? 'Error clearing history'
: (err.message || 'Failed to clear history'), 'error');
return false;
});
}
export function init(root, ctx) {
const els = elements(root);
const win = root.ownerDocument.defaultView;
const on = { signal: ctx.signal };
Object.assign(ctx.state, { els: els, records: [], filtered: [], page: 1, searchTimer: null });
els.refresh.addEventListener('click', function() { load(ctx); }, on);
els.clear.addEventListener('click', function() { clearHistory(ctx); }, on);
els.prev.addEventListener('click', function() {
if (ctx.state.page > 1) { ctx.state.page--; render(ctx); }
}, on);
els.next.addEventListener('click', function() {
if (ctx.state.page * PAGE_SIZE < ctx.state.filtered.length) { ctx.state.page++; render(ctx); }
}, on);
[els.plugin, els.type, els.status].forEach(function(select) {
select.addEventListener('change', function() { applyFilters(ctx); }, on);
});
els.search.addEventListener('input', function() {
win.clearTimeout(ctx.state.searchTimer);
ctx.state.searchTimer = win.setTimeout(function() {
ctx.state.searchTimer = null;
applyFilters(ctx);
}, SEARCH_DELAY_MS);
}, on);
loadPluginFilter(ctx);
load(ctx);
}
export function destroy(root, ctx) {
if (ctx.state.searchTimer) {
root.ownerDocument.defaultView.clearTimeout(ctx.state.searchTimer);
ctx.state.searchTimer = null;
}
}
@@ -0,0 +1,191 @@
/*
* pages/raw-json.js -- the Config Editor tab (templates/v3/partials/raw_json.html):
* plain-text editors for config.json and config_secrets.json.
*
* The buttons carry data-action ("format", "validate", "save") and
* data-editor ("main", "secrets"); one delegated listener on the page root
* handles all six. Validation messages are built with textContent, so a
* parser message that quotes the user's text stays text.
*
* Saves are writes: they are not cancelled when the page is swapped away,
* so the server always finishes and the result is still reported.
*
* The old globals (formatJson, manualValidateJson, validateJSON,
* saveMainConfig, saveSecretsConfig) are deprecated aliases made in
* core/boot.js; they forward to the exports at the bottom of this file.
*/
const EDITORS = {
main: {
editor: 'main-config-editor', validation: 'main-config-validation',
url: '/api/v3/config/raw/main', file: 'config.json',
},
secrets: {
editor: 'secrets-config-editor', validation: 'secrets-config-validation',
url: '/api/v3/config/raw/secrets', file: 'config_secrets.json',
},
};
// The mounted page, for the deprecated aliases (boot.js).
let active = null;
function el(doc, tag, className, text) {
const node = doc.createElement(tag);
if (className) node.className = className;
if (text !== undefined) node.textContent = text;
return node;
}
function parseError(text) {
try {
JSON.parse(text);
return null;
} catch (error) {
return error;
}
}
function parts(ctx, kind) {
const spec = EDITORS[kind];
if (!spec) return null;
const textarea = ctx.root.querySelector('#' + spec.editor);
const validation = ctx.root.querySelector('#' + spec.validation);
return textarea && validation ? { spec: spec, textarea: textarea, validation: validation } : null;
}
/** The short "Valid JSON" / "Invalid JSON: ..." line under an editor. True when valid. */
export function validate(ctx, kind) {
const p = parts(ctx, kind);
if (!p) return true;
const doc = ctx.root.ownerDocument;
const error = parseError(p.textarea.value);
const line = error
? el(doc, 'span', 'text-red-600')
: el(doc, 'span', 'text-green-600');
line.append(el(doc, 'i', (error ? 'fas fa-times-circle' : 'fas fa-check-circle') + ' mr-1'),
doc.createTextNode(error ? 'Invalid JSON: ' + error.message : 'Valid JSON'));
p.validation.replaceChildren(line);
return !error;
}
/** The Validate button: a detailed box, and a notification. */
export function validateDetailed(ctx, kind) {
const p = parts(ctx, kind);
if (!p) return false;
const doc = ctx.root.ownerDocument;
const error = parseError(p.textarea.value);
const box = el(doc, 'div', error
? 'p-3 bg-red-50 border border-red-200 rounded-md'
: 'p-3 bg-green-50 border border-green-200 rounded-md');
const row = el(doc, 'div', 'flex items-start');
const text = el(doc, 'div');
const detail = el(doc, 'div', error ? 'text-sm text-red-700 mt-1' : 'text-sm text-green-700 mt-1');
if (error) {
row.append(el(doc, 'i', 'fas fa-times-circle text-red-600 text-xl mr-3 mt-1'));
text.append(el(doc, 'div', 'font-semibold text-red-800', '✗ Invalid JSON syntax'));
detail.append(el(doc, 'strong', null, 'Error:'), doc.createTextNode(' ' + error.message));
} else {
row.append(el(doc, 'i', 'fas fa-check-circle text-green-600 text-xl mr-3 mt-1'));
text.append(el(doc, 'div', 'font-semibold text-green-800', '✓ JSON is valid!'));
['✓ Valid JSON syntax', '✓ Proper structure', '✓ No syntax errors detected'].forEach(function(line, i) {
if (i) detail.append(el(doc, 'br'));
detail.append(doc.createTextNode(line));
});
}
text.append(detail);
row.append(text);
box.append(row);
p.validation.replaceChildren(box);
ctx.notify(error ? 'JSON validation failed: ' + error.message : 'JSON validation successful!',
error ? 'error' : 'success');
return !error;
}
/** The Format button: re-indent valid JSON by four spaces. */
export function format(ctx, kind) {
const p = parts(ctx, kind);
if (!p) return false;
try {
p.textarea.value = JSON.stringify(JSON.parse(p.textarea.value), null, 4);
validate(ctx, kind);
ctx.notify('JSON formatted successfully!', 'success');
return true;
} catch (error) {
validate(ctx, kind);
ctx.notify('Cannot format invalid JSON: ' + error.message, 'error');
return false;
}
}
/** The Save button: validate, then post the parsed file. */
export function save(ctx, kind) {
const p = parts(ctx, kind);
if (!p) return Promise.resolve(false);
if (!validate(ctx, kind)) {
ctx.notify('Invalid JSON! Please fix errors before saving.', 'error');
return Promise.resolve(false);
}
const config = JSON.parse(p.textarea.value);
return ctx.api.post(p.spec.url, config).then(function() {
ctx.notify(p.spec.file + ' saved successfully!', 'success');
return true;
}).catch(function(err) {
if (err && err.loginRequired) return false;
ctx.notify('Error saving ' + p.spec.file + ': ' + ((err && err.message) || 'An error occurred'), 'error');
return false;
});
}
const ACTIONS = { format: format, validate: validateDetailed, save: save };
export function init(root, ctx) {
const on = { signal: ctx.signal };
root.addEventListener('click', function(event) {
const button = event.target.closest('button[data-action][data-editor]');
if (!button || !root.contains(button)) return;
const action = ACTIONS[button.dataset.action];
if (action) action(ctx, button.dataset.editor);
}, on);
Object.keys(EDITORS).forEach(function(kind) {
const p = parts(ctx, kind);
if (!p) return;
p.textarea.addEventListener('input', function() { validate(ctx, kind); }, on);
validate(ctx, kind);
});
active = ctx;
}
export function destroy(root, ctx) {
if (active === ctx) active = null;
}
// ── the old globals, kept as deprecated aliases (boot.js) ────────────────────
// They took element ids; map those back to an editor.
function kindOf(editorId) {
return Object.keys(EDITORS).find(function(kind) { return EDITORS[kind].editor === editorId; });
}
/** window.formatJson(editorId, validationDivId) */
export function formatJson(editorId) {
return active ? format(active, kindOf(editorId)) : false;
}
/** window.manualValidateJson(editorId, validationDivId) */
export function manualValidateJson(editorId) {
return active ? validateDetailed(active, kindOf(editorId)) : false;
}
/** window.validateJSON(editorId, validationDivId): true when valid. */
export function validateJSON(editorId) {
return active ? validate(active, kindOf(editorId)) : true;
}
/** window.saveMainConfig() */
export function saveMainConfig() {
return active ? save(active, 'main') : Promise.resolve(false);
}
/** window.saveSecretsConfig() */
export function saveSecretsConfig() {
return active ? save(active, 'secrets') : Promise.resolve(false);
}
@@ -12,7 +12,9 @@
* orderInputId: 'vegas_plugin_order_value', // hidden input, JSON array of ids
* excludedInputId: 'vegas_excluded_plugins_value', // optional: adds an
* // include-checkbox per row; unchecked ids collect here (JSON array)
* showVegasModeBadge: true // optional: Scroll/Pause/Excluded badge
* showVegasModeBadge: true, // optional: Scroll/Pause/Excluded badge
* signal: ctx.signal // optional: AbortSignal that cancels
* // the plugin-list request (a page module's ctx.signal)
* });
*
* The container re-renders from /api/v3/plugins/installed each init; the
@@ -97,7 +99,7 @@
});
}
fetch('/api/v3/plugins/installed')
fetch('/api/v3/plugins/installed', { signal: options.signal })
.then(response => response.json())
.then(data => {
const allPlugins = (data.data && data.data.plugins) || data.plugins || [];
@@ -216,6 +218,8 @@
syncInputs();
})
.catch(error => {
// The page was swapped away (options.signal): nothing to draw.
if (error && error.name === 'AbortError') return;
console.error('Error fetching plugins:', error);
const err = document.createElement('p');
err.className = 'text-sm text-red-500';
@@ -1,5 +1,9 @@
{% import 'v3/partials/_macros.html' as ui %}
<div class="space-y-6" id="backup-restore-root">
{# No inline script: static/v3/js/pages/backup-restore.js runs this page. Its
buttons name an action (data-action) for one delegated listener. The page
registry (static/v3/js/core/registry.js) starts it when this root appears
and stops it when the partial is swapped away. #}
<div class="space-y-6" id="backup-restore-root" data-page="backup-restore">
<!-- Security warning -->
<div class="bg-red-50 border border-red-200 rounded-lg p-4">
@@ -28,7 +32,7 @@
Download a single ZIP with all of your settings so you can restore it later.
</p>
</div>
<button onclick="exportBackup()" id="export-backup-btn"
<button type="button" data-action="export" id="export-backup-btn"
class="inline-flex items-center px-4 py-2 border border-transparent text-sm font-medium rounded-md text-white bg-blue-600 hover:bg-blue-700">
<i class="fas fa-download mr-2"></i>
Download backup
@@ -58,7 +62,7 @@
</div>
<div>
<button onclick="validateRestoreFile()" id="validate-restore-btn"
<button type="button" data-action="inspect" id="validate-restore-btn"
class="inline-flex items-center px-3 py-2 border border-gray-300 text-sm font-medium rounded-md text-gray-700 bg-white hover:bg-gray-50">
<i class="fas fa-check-circle mr-2"></i>
Inspect file
@@ -80,12 +84,12 @@
</div>
<div class="mt-4 flex gap-2">
<button onclick="runRestore()" id="run-restore-btn"
<button type="button" data-action="restore" id="run-restore-btn"
class="inline-flex items-center px-4 py-2 border border-transparent text-sm font-medium rounded-md text-white bg-green-600 hover:bg-green-700">
<i class="fas fa-upload mr-2"></i>
Restore now
</button>
<button onclick="clearRestore()"
<button type="button" data-action="cancel"
class="inline-flex items-center px-3 py-2 border border-gray-300 text-sm font-medium rounded-md text-gray-700 bg-white hover:bg-gray-50">
Cancel
</button>
@@ -104,7 +108,7 @@
<h2 class="text-lg font-semibold text-gray-900">Backup history</h2>
<p class="mt-1 text-sm text-gray-600">Previously exported backups stored on this device.</p>
</div>
<button onclick="loadBackupList()"
<button type="button" data-action="refresh"
class="inline-flex items-center px-3 py-2 border border-gray-300 text-sm font-medium rounded-md text-gray-700 bg-white hover:bg-gray-50">
<i class="fas fa-sync-alt mr-2"></i>
Refresh
@@ -115,237 +119,3 @@
</div>
</div>
<script>
(function () {
let inspectedFile = null;
function notify(message, kind) {
showNotification(message, kind || 'info');
}
function formatSize(bytes) {
if (!bytes) return '0 B';
const units = ['B', 'KB', 'MB', 'GB'];
let i = 0, size = bytes;
while (size >= 1024 && i < units.length - 1) { size /= 1024; i++; }
return size.toFixed(i === 0 ? 0 : 1) + ' ' + units[i];
}
function escapeHtml(value) { return window.LEDEscape.html(value); }
async function loadPreview() {
const el = document.getElementById('export-preview');
try {
const res = await fetch('/api/v3/backup/preview');
const payload = await res.json();
if (payload.status !== 'success') throw new Error(payload.message || 'Preview failed');
const d = payload.data || {};
el.innerHTML = `
<ul class="list-disc pl-5 space-y-1">
<li>Main config: <strong>${d.has_config ? 'yes' : 'no'}</strong></li>
<li>Secrets: <strong>${d.has_secrets ? 'yes' : 'no'}</strong></li>
<li>WiFi config: <strong>${d.has_wifi ? 'yes' : 'no'}</strong></li>
<li>User fonts: <strong>${(d.user_fonts || []).length}</strong> ${d.user_fonts && d.user_fonts.length ? '(' + d.user_fonts.map(escapeHtml).join(', ') + ')' : ''}</li>
<li>Plugin image uploads: <strong>${d.plugin_uploads || 0}</strong> file(s)</li>
<li>Installed plugins: <strong>${(d.plugins || []).length}</strong></li>
</ul>`;
} catch (err) {
el.textContent = 'Could not load preview: ' + err.message;
}
}
async function loadBackupList() {
const el = document.getElementById('backup-history');
el.textContent = 'Loading…';
try {
const res = await fetch('/api/v3/backup/list');
const payload = await res.json();
if (payload.status !== 'success') throw new Error(payload.message || 'List failed');
const entries = payload.data || [];
if (!entries.length) {
el.innerHTML = '<p>No backups have been created yet.</p>';
return;
}
el.innerHTML = `
<table class="min-w-full divide-y divide-gray-200">
<thead>
<tr>
<th class="text-left py-2">Filename</th>
<th class="text-left py-2">Size</th>
<th class="text-left py-2">Created</th>
<th></th>
</tr>
</thead>
<tbody class="divide-y divide-gray-100">
${entries.map(e => `
<tr>
<td class="py-2 font-mono text-xs">${escapeHtml(e.filename)}</td>
<td class="py-2">${formatSize(e.size)}</td>
<td class="py-2">${escapeHtml(e.created_at)}</td>
<td class="py-2 text-right space-x-2">
<a href="/api/v3/backup/download/${encodeURIComponent(e.filename)}"
class="text-blue-600 hover:underline">Download</a>
<button data-filename="${escapeHtml(e.filename)}"
class="text-red-600 hover:underline backup-delete-btn">Delete</button>
</td>
</tr>
`).join('')}
</tbody>
</table>`;
el.querySelectorAll('.backup-delete-btn').forEach(btn => {
btn.addEventListener('click', () => deleteBackup(btn.dataset.filename));
});
} catch (err) {
el.textContent = 'Could not load backups: ' + err.message;
}
}
async function exportBackup() {
const btn = document.getElementById('export-backup-btn');
btn.disabled = true;
btn.innerHTML = '<i class="fas fa-spinner fa-spin mr-2"></i>Creating…';
try {
const res = await fetch('/api/v3/backup/export', { method: 'POST' });
const payload = await res.json();
if (payload.status !== 'success') throw new Error(payload.message || 'Export failed');
notify('Backup created: ' + payload.filename, 'success');
// Trigger browser download immediately.
window.location.href = '/api/v3/backup/download/' + encodeURIComponent(payload.filename);
await loadBackupList();
} catch (err) {
notify('Export failed: ' + err.message, 'error');
} finally {
btn.disabled = false;
btn.innerHTML = '<i class="fas fa-download mr-2"></i>Download backup';
}
}
async function deleteBackup(filename) {
if (!confirm('Delete ' + filename + '?')) return;
try {
const res = await fetch('/api/v3/backup/' + encodeURIComponent(filename), { method: 'DELETE' });
const payload = await res.json();
if (payload.status !== 'success') throw new Error(payload.message || 'Delete failed');
notify('Backup deleted', 'success');
await loadBackupList();
} catch (err) {
notify('Delete failed: ' + err.message, 'error');
}
}
async function validateRestoreFile() {
const input = document.getElementById('restore-file-input');
if (!input.files || !input.files[0]) {
notify('Choose a backup file first', 'error');
return;
}
const file = input.files[0];
const fd = new FormData();
fd.append('backup_file', file);
try {
const res = await fetch('/api/v3/backup/validate', { method: 'POST', body: fd });
const payload = await res.json();
if (payload.status !== 'success') throw new Error(payload.message || 'Validation failed');
inspectedFile = file;
renderRestorePreview(payload.data);
} catch (err) {
notify('Invalid backup: ' + err.message, 'error');
}
}
function renderRestorePreview(manifest) {
const wrap = document.getElementById('restore-preview');
const body = document.getElementById('restore-preview-body');
const detected = manifest.detected_contents || [];
const plugins = manifest.plugins || [];
body.innerHTML = `
<div><strong>Created:</strong> ${escapeHtml(manifest.created_at || 'unknown')}</div>
<div><strong>Source host:</strong> ${escapeHtml(manifest.hostname || 'unknown')}</div>
<div><strong>LEDMatrix version:</strong> ${escapeHtml(manifest.ledmatrix_version || 'unknown')}</div>
<div><strong>Includes:</strong> ${detected.length ? detected.map(escapeHtml).join(', ') : '(nothing detected)'}</div>
<div><strong>Plugins referenced:</strong> ${plugins.length ? plugins.map(p => escapeHtml(p.plugin_id)).join(', ') : 'none'}</div>
`;
wrap.classList.remove('hidden');
}
function clearRestore() {
inspectedFile = null;
document.getElementById('restore-preview').classList.add('hidden');
document.getElementById('restore-result').classList.add('hidden');
document.getElementById('restore-file-input').value = '';
}
async function runRestore() {
if (!inspectedFile) {
notify('Inspect the file before restoring', 'error');
return;
}
if (!confirm('Restore from this backup? Current configuration will be overwritten.')) return;
const options = {
restore_config: document.getElementById('opt-config').checked,
restore_secrets: document.getElementById('opt-secrets').checked,
restore_wifi: document.getElementById('opt-wifi').checked,
restore_fonts: document.getElementById('opt-fonts').checked,
restore_plugin_uploads: document.getElementById('opt-plugin-uploads').checked,
reinstall_plugins: document.getElementById('opt-reinstall').checked,
};
const fd = new FormData();
fd.append('backup_file', inspectedFile);
fd.append('options', JSON.stringify(options));
const btn = document.getElementById('run-restore-btn');
btn.disabled = true;
btn.innerHTML = '<i class="fas fa-spinner fa-spin mr-2"></i>Restoring…';
try {
const res = await fetch('/api/v3/backup/restore', { method: 'POST', body: fd });
const payload = await res.json();
if (payload.status !== 'success') {
const msgs = (payload.data?.errors || []).join('; ');
throw new Error(payload.message || msgs || 'Restore had errors');
}
const data = payload.data || {};
const hasPartial = (data.plugins_failed || []).length > 0 || (data.errors || []).length > 0;
const result = document.getElementById('restore-result');
result.className = (hasPartial
? 'bg-yellow-50 border-yellow-200 text-yellow-800'
: 'bg-green-50 border-green-200 text-green-800') + ' border rounded-md p-4';
result.classList.remove('hidden');
result.innerHTML = `
<h3 class="font-medium mb-2">${hasPartial ? 'Restore complete with warnings' : 'Restore complete'}</h3>
<div><strong>Restored:</strong> ${(data.restored || []).map(escapeHtml).join(', ') || 'none'}</div>
<div><strong>Skipped:</strong> ${(data.skipped || []).map(escapeHtml).join(', ') || 'none'}</div>
<div><strong>Plugins installed:</strong> ${(data.plugins_installed || []).map(escapeHtml).join(', ') || 'none'}</div>
<div><strong>Plugins failed:</strong> ${(data.plugins_failed || []).map(p => escapeHtml(p.plugin_id + ' (' + p.error + ')')).join(', ') || 'none'}</div>
<div><strong>Errors:</strong> ${(data.errors || []).map(escapeHtml).join('; ') || 'none'}</div>
${((data.restored || []).length || (data.plugins_installed || []).length) ? '<p class="mt-2">Restart the display service to apply all changes.</p>' : ''}
`;
notify(hasPartial ? 'Restore complete with warnings' : 'Restore complete', hasPartial ? 'warning' : 'success');
} catch (err) {
notify('Restore failed: ' + err.message, 'error');
} finally {
btn.disabled = false;
btn.innerHTML = '<i class="fas fa-upload mr-2"></i>Restore now';
}
}
// Expose handlers to inline onclick attributes.
window.exportBackup = exportBackup;
window.loadBackupList = loadBackupList;
window.validateRestoreFile = validateRestoreFile;
window.clearRestore = clearRestore;
window.runRestore = runRestore;
// Clear inspection state whenever the user picks a new file.
document.getElementById('restore-file-input').addEventListener('change', function () {
inspectedFile = null;
document.getElementById('restore-preview').classList.add('hidden');
document.getElementById('restore-result').classList.add('hidden');
});
// Initial load.
loadPreview();
loadBackupList();
})();
</script>
@@ -1,5 +1,8 @@
{% import 'v3/partials/_macros.html' as ui %}
<div class="bg-white rounded-lg shadow p-6">
{# No inline script: static/v3/js/pages/durations.js starts the rotation-order
list. The page registry (static/v3/js/core/registry.js) starts it when this
root appears and stops it when the partial is swapped away. #}
<div class="bg-white rounded-lg shadow p-6" data-page="durations">
<div class="border-b border-gray-200 pb-4 mb-6">
<h2 class="text-lg font-semibold text-gray-900">Rotation &amp; Durations</h2>
<p class="mt-1 text-sm text-gray-600">Set the order plugins rotate on the display and how long each screen is shown. Durations are in seconds.</p>
@@ -77,34 +80,3 @@
</div>
</form>
</div>
<script>
(function () {
// Shared drag-and-drop plugin list (static/v3/js/widgets/plugin-order-list.js,
// same module the Vegas Scroll section uses).
function initRotationOrderList(attempt) {
const container = document.getElementById('rotation_plugin_order');
if (!container) return;
if (!window.PluginOrderList) {
// Widget script is deferred; retry briefly, then surface a real
// error instead of showing "Loading…" forever.
if ((attempt || 0) < 50) {
setTimeout(function() { initRotationOrderList((attempt || 0) + 1); }, 100);
} else {
container.textContent = 'Could not load the reorder widget — reload the page to try again.';
container.className = 'text-sm text-red-500';
}
return;
}
window.PluginOrderList.init({
containerId: 'rotation_plugin_order',
orderInputId: 'rotation_plugin_order_value'
});
}
if (document.readyState === 'loading') {
document.addEventListener('DOMContentLoaded', initRotationOrderList);
} else {
initRotationOrderList();
}
}());
</script>
@@ -1,4 +1,7 @@
<div class="bg-white rounded-lg shadow p-6">
{# No inline script: static/v3/js/pages/operation-history.js runs this page.
The page registry (static/v3/js/core/registry.js) starts it when this root
appears and stops it when the partial is swapped away. #}
<div class="bg-white rounded-lg shadow p-6" data-page="operation-history">
<div class="border-b border-gray-200 pb-4 mb-6">
<h2 class="text-lg font-semibold text-gray-900">Operation History</h2>
<p class="mt-1 text-sm text-gray-600">View history of plugin operations and configuration changes for debugging and auditing.</p>
@@ -39,12 +42,12 @@
<div class="flex items-center space-x-2">
<!-- Refresh -->
<button id="refresh-history-btn" class="btn bg-blue-600 hover:bg-blue-700 text-white px-3 py-1 rounded text-sm">
<button type="button" id="refresh-history-btn" class="btn bg-blue-600 hover:bg-blue-700 text-white px-3 py-1 rounded text-sm">
<i class="fas fa-sync-alt mr-1"></i>Refresh
</button>
<!-- Clear History -->
<button id="clear-history-btn" class="btn bg-red-600 hover:bg-red-700 text-white px-3 py-1 rounded text-sm">
<button type="button" id="clear-history-btn" class="btn bg-red-600 hover:bg-red-700 text-white px-3 py-1 rounded text-sm">
<i class="fas fa-trash mr-1"></i>Clear
</button>
</div>
@@ -81,307 +84,12 @@
Showing <span id="history-start">0</span> to <span id="history-end">0</span> of <span id="history-total">0</span> operations
</div>
<div class="flex space-x-2">
<button id="history-prev-btn" class="btn bg-gray-600 hover:bg-gray-700 text-white px-3 py-1 rounded text-sm disabled:opacity-50" disabled>
<button type="button" id="history-prev-btn" class="btn bg-gray-600 hover:bg-gray-700 text-white px-3 py-1 rounded text-sm disabled:opacity-50" disabled>
<i class="fas fa-chevron-left mr-1"></i>Previous
</button>
<button id="history-next-btn" class="btn bg-gray-600 hover:bg-gray-700 text-white px-3 py-1 rounded text-sm disabled:opacity-50" disabled>
<button type="button" id="history-next-btn" class="btn bg-gray-600 hover:bg-gray-700 text-white px-3 py-1 rounded text-sm disabled:opacity-50" disabled>
Next<i class="fas fa-chevron-right ml-1"></i>
</button>
</div>
</div>
</div>
<script>
(function() {
let currentPage = 1;
const pageSize = 50;
let allHistory = [];
let filteredHistory = [];
// Normalize status values so both server-side conventions match filter values.
// The server may return "completed" or "success" for successful operations,
// and "error" or "failed" for failures.
function normalizeStatus(status) {
const s = (status || '').toLowerCase();
if (s === 'completed' || s === 'success') return 'success';
if (s === 'error' || s === 'failed') return 'failed';
return s;
}
// Load operation history
async function loadHistory() {
try {
const response = await fetch('/api/v3/plugins/operation/history?limit=1000');
const data = await response.json();
if (data.status === 'success') {
allHistory = data.data || [];
applyFilters();
} else {
console.error('Failed to load history:', data.message);
showError('Failed to load operation history');
}
} catch (error) {
console.error('Error loading history:', error);
showError('Error loading operation history');
}
}
// Apply filters
function applyFilters() {
const pluginFilter = document.getElementById('history-plugin-filter')?.value || '';
const typeFilter = document.getElementById('history-type-filter')?.value || '';
const statusFilter = document.getElementById('history-status-filter')?.value || '';
const searchTerm = (document.getElementById('history-search')?.value || '').toLowerCase();
filteredHistory = allHistory.filter(record => {
if (pluginFilter && record.plugin_id !== pluginFilter) return false;
if (typeFilter && record.operation_type !== typeFilter) return false;
if (statusFilter && normalizeStatus(record.status) !== statusFilter) return false;
if (searchTerm) {
const searchable = [
record.operation_type,
record.plugin_id,
record.status,
record.user,
record.error,
JSON.stringify(record.details || {})
].join(' ').toLowerCase();
if (!searchable.includes(searchTerm)) return false;
}
return true;
});
renderHistory();
}
// Render history table
function renderHistory() {
const tbody = document.getElementById('history-table-body');
if (!tbody) return;
const start = (currentPage - 1) * pageSize;
const end = Math.min(start + pageSize, filteredHistory.length);
const pageData = filteredHistory.slice(start, end);
if (pageData.length === 0) {
tbody.innerHTML = `
<tr>
<td colspan="6" class="px-6 py-4 text-center text-gray-500">
No operations found
</td>
</tr>
`;
return;
}
tbody.innerHTML = pageData.map(record => {
const timestamp = new Date(record.timestamp || record.created_at || Date.now());
const timeStr = timestamp.toLocaleString();
const normalized = normalizeStatus(record.status);
const statusClass = getStatusClass(normalized);
const operationType = record.operation_type || 'unknown';
const pluginId = record.plugin_id || '-';
const user = record.user || '-';
const error = record.error ? `<div class="text-red-600 text-xs mt-1">${escapeHtml(record.error)}</div>` : '';
// Build a concise details summary
let detailsSummary = '';
const d = record.details;
if (d) {
const parts = [];
if (d.version) parts.push(`v${d.version}`);
if (d.branch) parts.push(`branch: ${d.branch}`);
if (d.commit) parts.push(`commit: ${String(d.commit).substring(0, 7)}`);
if (d.previous_commit && d.commit && d.previous_commit !== d.commit) {
parts.push(`from: ${String(d.previous_commit).substring(0, 7)}`);
}
if (d.preserve_config !== undefined) parts.push(d.preserve_config ? 'config preserved' : 'config removed');
detailsSummary = parts.length > 0 ? `<span class="text-xs text-gray-500">${escapeHtml(parts.join(' | '))}</span>` : '';
}
return `
<tr class="hover:bg-gray-50">
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-900">${escapeHtml(timeStr)}</td>
<td class="px-6 py-4 whitespace-nowrap text-sm">
<span class="px-2 py-1 text-xs font-medium rounded ${getOperationTypeClass(operationType)}">
${escapeHtml(operationType)}
</span>
</td>
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-900">${escapeHtml(pluginId)}</td>
<td class="px-6 py-4 whitespace-nowrap text-sm">
<span class="px-2 py-1 text-xs font-medium rounded ${statusClass}">
${escapeHtml(normalized)}
</span>
</td>
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500">${escapeHtml(user)}</td>
<td class="px-6 py-4 text-sm text-gray-500">
${detailsSummary || '-'}
${error}
</td>
</tr>
`;
}).join('');
// Update pagination
document.getElementById('history-start').textContent = filteredHistory.length > 0 ? start + 1 : 0;
document.getElementById('history-end').textContent = end;
document.getElementById('history-total').textContent = filteredHistory.length;
// Update pagination buttons
const prevBtn = document.getElementById('history-prev-btn');
const nextBtn = document.getElementById('history-next-btn');
if (prevBtn) prevBtn.disabled = currentPage === 1;
if (nextBtn) nextBtn.disabled = end >= filteredHistory.length;
}
// Helper functions
function getStatusClass(status) {
if (status === 'success') return 'bg-green-100 text-green-800';
if (status === 'failed') return 'bg-red-100 text-red-800';
return 'bg-gray-100 text-gray-800';
}
function getOperationTypeClass(type) {
const typeLower = (type || '').toLowerCase();
if (typeLower === 'install') {
return 'bg-blue-100 text-blue-800';
} else if (typeLower === 'update') {
return 'bg-purple-100 text-purple-800';
} else if (typeLower === 'uninstall') {
return 'bg-red-100 text-red-800';
} else if (typeLower === 'enable') {
return 'bg-green-100 text-green-800';
} else if (typeLower === 'disable') {
return 'bg-yellow-100 text-yellow-800';
} else if (typeLower === 'configure') {
return 'bg-teal-100 text-teal-800';
}
return 'bg-gray-100 text-gray-800';
}
function escapeHtml(text) { return window.LEDEscape.html(text); }
function showError(message) {
showNotification(message, 'error');
}
// Populate plugin filter
async function populatePluginFilter() {
try {
// Use PluginAPI if available, otherwise fall back to direct fetch
let data;
if (window.PluginAPI && window.PluginAPI.getInstalledPlugins) {
const plugins = await window.PluginAPI.getInstalledPlugins();
data = { status: 'success', data: { plugins: plugins } };
} else {
const response = await fetch('/api/v3/plugins/installed');
data = await response.json();
}
if (data.status === 'success' && data.data) {
const select = document.getElementById('history-plugin-filter');
if (select) {
const plugins = (data.data.plugins || []).map(p => p.id).filter(Boolean);
plugins.forEach(pluginId => {
const option = document.createElement('option');
option.value = pluginId;
option.textContent = pluginId;
select.appendChild(option);
});
}
}
} catch (error) {
console.error('Error loading plugins for filter:', error);
}
}
// Event listeners
function setupEventListeners() {
const refreshBtn = document.getElementById('refresh-history-btn');
if (refreshBtn) {
refreshBtn.addEventListener('click', () => {
loadHistory();
});
}
const clearBtn = document.getElementById('clear-history-btn');
if (clearBtn) {
clearBtn.addEventListener('click', async () => {
if (confirm('Are you sure you want to clear the operation history? This cannot be undone.')) {
try {
const response = await fetch('/api/v3/plugins/operation/history', { method: 'DELETE' });
const data = await response.json();
if (data.status === 'success') {
allHistory = [];
applyFilters();
} else {
showError(data.message || 'Failed to clear history');
}
} catch (error) {
showError('Error clearing history');
}
}
});
}
const prevBtn = document.getElementById('history-prev-btn');
if (prevBtn) {
prevBtn.addEventListener('click', () => {
if (currentPage > 1) {
currentPage--;
renderHistory();
}
});
}
const nextBtn = document.getElementById('history-next-btn');
if (nextBtn) {
nextBtn.addEventListener('click', () => {
const start = (currentPage - 1) * pageSize;
const end = Math.min(start + pageSize, filteredHistory.length);
if (end < filteredHistory.length) {
currentPage++;
renderHistory();
}
});
}
// Filter change listeners
['history-plugin-filter', 'history-type-filter', 'history-status-filter'].forEach(id => {
const element = document.getElementById(id);
if (element) {
element.addEventListener('change', () => {
currentPage = 1;
applyFilters();
});
}
});
const searchInput = document.getElementById('history-search');
if (searchInput) {
let searchTimeout;
searchInput.addEventListener('input', () => {
clearTimeout(searchTimeout);
searchTimeout = setTimeout(() => {
currentPage = 1;
applyFilters();
}, 300);
});
}
}
// Initialize
if (document.readyState === 'loading') {
document.addEventListener('DOMContentLoaded', () => {
setupEventListeners();
populatePluginFilter();
loadHistory();
});
} else {
setupEventListeners();
populatePluginFilter();
loadHistory();
}
})();
</script>
+11 -222
View File
@@ -1,4 +1,8 @@
<div class="space-y-6">
{# No inline script: static/v3/js/pages/raw-json.js runs this page. Its buttons
name an action and an editor (data-action, data-editor) for one delegated
listener. The page registry (static/v3/js/core/registry.js) starts it when
this root appears and stops it when the partial is swapped away. #}
<div class="space-y-6" data-page="raw-json">
<!-- Config.json Editor -->
<div class="bg-white rounded-lg shadow p-6">
<div class="border-b border-gray-200 pb-4 mb-6">
@@ -8,17 +12,17 @@
<p class="mt-1 text-sm text-gray-600">{{ main_config_path }}</p>
</div>
<div class="flex gap-2">
<button onclick="formatJson('main-config-editor', 'main-config-validation')"
<button type="button" data-action="format" data-editor="main"
class="inline-flex items-center px-3 py-2 border border-gray-300 text-sm font-medium rounded-md text-gray-700 bg-white hover:bg-gray-50">
<i class="fas fa-align-left mr-2"></i>
Format JSON
</button>
<button onclick="manualValidateJson('main-config-editor', 'main-config-validation')"
<button type="button" data-action="validate" data-editor="main"
class="inline-flex items-center px-3 py-2 border border-transparent text-sm font-medium rounded-md text-white bg-yellow-600 hover:bg-yellow-700">
<i class="fas fa-check-circle mr-2"></i>
Validate JSON
</button>
<button onclick="saveMainConfig()"
<button type="button" data-action="save" data-editor="main"
class="inline-flex items-center px-4 py-2 border border-transparent text-sm font-medium rounded-md text-white bg-blue-600 hover:bg-blue-700">
<i class="fas fa-save mr-2"></i>
Save
@@ -59,17 +63,17 @@
<p class="mt-1 text-sm text-gray-600">{{ secrets_config_path }}</p>
</div>
<div class="flex gap-2">
<button onclick="formatJson('secrets-config-editor', 'secrets-config-validation')"
<button type="button" data-action="format" data-editor="secrets"
class="inline-flex items-center px-3 py-2 border border-gray-300 text-sm font-medium rounded-md text-gray-700 bg-white hover:bg-gray-50">
<i class="fas fa-align-left mr-2"></i>
Format JSON
</button>
<button onclick="manualValidateJson('secrets-config-editor', 'secrets-config-validation')"
<button type="button" data-action="validate" data-editor="secrets"
class="inline-flex items-center px-3 py-2 border border-transparent text-sm font-medium rounded-md text-white bg-yellow-600 hover:bg-yellow-700">
<i class="fas fa-check-circle mr-2"></i>
Validate JSON
</button>
<button onclick="saveSecretsConfig()"
<button type="button" data-action="save" data-editor="secrets"
class="inline-flex items-center px-4 py-2 border border-transparent text-sm font-medium rounded-md text-white bg-blue-600 hover:bg-blue-700">
<i class="fas fa-save mr-2"></i>
Save
@@ -101,218 +105,3 @@
</div>
</div>
</div>
<script>
// Format JSON with proper indentation
function formatJson(editorId, validationDivId) {
const textarea = document.getElementById(editorId);
const jsonText = textarea.value;
try {
const parsed = JSON.parse(jsonText);
const formatted = JSON.stringify(parsed, null, 4);
textarea.value = formatted;
// Auto-validate after formatting
validateJSON(editorId, validationDivId);
showNotification('JSON formatted successfully!', 'success');
} catch (error) {
showNotification('Cannot format invalid JSON: ' + error.message, 'error');
validateJSON(editorId, validationDivId);
}
}
// Manual validation with detailed feedback
function manualValidateJson(editorId, validationDivId) {
const textarea = document.getElementById(editorId);
const validation = document.getElementById(validationDivId);
const jsonText = textarea.value;
if (!textarea || !validation) return;
try {
const parsed = JSON.parse(jsonText);
validation.innerHTML = `
<div class="p-3 bg-green-50 border border-green-200 rounded-md">
<div class="flex items-start">
<i class="fas fa-check-circle text-green-600 text-xl mr-3 mt-1"></i>
<div>
<div class="font-semibold text-green-800">✓ JSON is valid!</div>
<div class="text-sm text-green-700 mt-1">
✓ Valid JSON syntax<br>
✓ Proper structure<br>
✓ No syntax errors detected
</div>
</div>
</div>
</div>
`;
showNotification('JSON validation successful!', 'success');
} catch (e) {
validation.innerHTML = `
<div class="p-3 bg-red-50 border border-red-200 rounded-md">
<div class="flex items-start">
<i class="fas fa-times-circle text-red-600 text-xl mr-3 mt-1"></i>
<div>
<div class="font-semibold text-red-800">✗ Invalid JSON syntax</div>
<div class="text-sm text-red-700 mt-1">
<strong>Error:</strong> ${window.LEDEscape.html(e.message)}
</div>
</div>
</div>
</div>
`;
showNotification('JSON validation failed: ' + e.message, 'error');
}
}
// Auto-validate JSON as user types (simple version)
function validateJSON(editor, validationDiv) {
const textarea = document.getElementById(editor);
const validation = document.getElementById(validationDiv);
if (!textarea || !validation) return true;
try {
JSON.parse(textarea.value);
validation.innerHTML = '<span class="text-green-600"><i class="fas fa-check-circle mr-1"></i>Valid JSON</span>';
return true;
} catch (e) {
validation.innerHTML = '<span class="text-red-600"><i class="fas fa-times-circle mr-1"></i>Invalid JSON: ' + e.message + '</span>';
return false;
}
}
// Auto-validate on input
document.getElementById('main-config-editor')?.addEventListener('input', function() {
validateJSON('main-config-editor', 'main-config-validation');
});
document.getElementById('secrets-config-editor')?.addEventListener('input', function() {
validateJSON('secrets-config-editor', 'secrets-config-validation');
});
// Initial validation
setTimeout(() => {
validateJSON('main-config-editor', 'main-config-validation');
validateJSON('secrets-config-editor', 'secrets-config-validation');
}, 100);
function saveMainConfig() {
const textarea = document.getElementById('main-config-editor');
// Validate JSON first
if (!validateJSON('main-config-editor', 'main-config-validation')) {
showNotification('Invalid JSON! Please fix errors before saving.', 'error');
return;
}
try {
const config = JSON.parse(textarea.value);
// Save via API
fetch('/api/v3/config/raw/main', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(config)
})
.then(async response => {
// Store status and statusText before parsing
const status = response.status;
const statusText = response.statusText;
// Try to parse JSON response
let data;
try {
data = await response.json();
} catch (parseError) {
// If JSON parsing fails, throw generic HTTP error
throw new Error(`HTTP ${status}: ${statusText}`);
}
// Handle non-OK responses
if (!response.ok) {
// Extract specific error message from API response if available
const errorMessage = data.message || data.status || statusText;
throw new Error(errorMessage);
}
// Handle successful responses
if (data.status === 'success') {
showNotification('config.json saved successfully!', 'success');
} else {
showNotification('Error saving config.json: ' + (data.message || 'Unknown error'), 'error');
}
})
.catch(error => {
// Preserve the error message that was intentionally thrown
showNotification('Error saving config.json: ' + (error.message || 'An error occurred'), 'error');
});
} catch (e) {
showNotification('Invalid JSON: ' + e.message, 'error');
}
}
function saveSecretsConfig() {
const textarea = document.getElementById('secrets-config-editor');
// Validate JSON first
if (!validateJSON('secrets-config-editor', 'secrets-config-validation')) {
showNotification('Invalid JSON! Please fix errors before saving.', 'error');
return;
}
try {
const config = JSON.parse(textarea.value);
// Save via API
fetch('/api/v3/config/raw/secrets', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(config)
})
.then(async response => {
// Store status and statusText before parsing
const status = response.status;
const statusText = response.statusText;
// Try to parse JSON response
let data;
try {
data = await response.json();
} catch (parseError) {
// If JSON parsing fails, throw generic HTTP error
throw new Error(`HTTP ${status}: ${statusText}`);
}
// Handle non-OK responses
if (!response.ok) {
// Extract specific error message from API response if available
const errorMessage = data.message || data.status || statusText;
throw new Error(errorMessage);
}
// Handle successful responses
if (data.status === 'success') {
showNotification('config_secrets.json saved successfully!', 'success');
} else {
showNotification('Error saving config_secrets.json: ' + (data.message || 'Unknown error'), 'error');
}
})
.catch(error => {
// Preserve the error message that was intentionally thrown
showNotification('Error saving config_secrets.json: ' + (error.message || 'An error occurred'), 'error');
});
} catch (e) {
showNotification('Invalid JSON: ' + e.message, 'error');
}
}
// showNotification is provided by the notification widget (notification.js)
// No local definition needed - uses window.showNotification from the widget
</script>