Compare commits

...
Author SHA1 Message Date
ChuckandClaude Opus 5 e6249dcc7e fix(service): survive corrupt health cache and clean exits
Three independent failure modes that each end with a dark panel and no
automatic recovery.

1. PluginHealthTracker._load_health_state returned the cached value
   verbatim. If that value is not a dict, every caller raises
   AttributeError: 'list' object has no attribute 'get' — during
   DisplayController.__init__, so the process dies before the display
   loop starts. systemd restarts it, the same bad entry is read back
   from disk, and it dies again: an unattended restart loop that
   survives reboots because the cause is persisted. Observed in the
   field with plugin_health:<id> holding an unrelated plugin's list
   payload. Now non-dict entries are discarded with a warning and the
   defaults are rebuilt.

2. ledmatrix.service used Restart=on-failure, so any exit with status 0
   left the unit stopped and the panel dark indefinitely — systemd
   treats it as success and never brings it back. Restart=always.

3. ledmatrix-wifi-monitor.service used StandardOutput=syslog, which
   systemd has marked obsolete; it warns and rewrites it to journal on
   every load.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 19:04:57 -04:00
9083df9f5c fix(pixlet): resolve the release tag correctly when downloading (#461)
* fix(pixlet): resolve the release tag correctly when downloading

Starlark apps render through the pixlet binary, and the installer that
fetches it silently produced nothing, so every app failed with "Pixlet
not available - Starlark apps will not work".

Two compounding defects:

The version lookup parsed the wrong token. GitHub returns the release
JSON on a single line, so `grep '"tag_name"'` matches the whole document
and the greedy `sed 's/.*"([^"]+)".*/\1/'` captures the LAST quoted
string in it. That resolved to "mentions_count", giving a download URL
for a release that does not exist. The `[ -z "$PIXLET_VERSION" ]`
fallback never fired, because the value was not empty -- just wrong.

And `curl -L -o` without `-f` writes a 404 body to the file and exits 0,
so the download was reported as successful and the first sign of trouble
was tar complaining "not in gzip format" about a page of HTML:

    → Downloading linux-arm64...
      Extracting...
    gzip: stdin: not in gzip format
    ✗ Failed to extract archive: .../pixlet_mentions_count_linux-arm64.tar.gz
    Download complete: 0/1 succeeded

Now the tag field is matched directly and the value taken from it, and
the result is checked for a version shape rather than merely being
non-empty -- a wrong-but-non-empty value is exactly what made this
silent. curl gets -f so an HTTP error is a failure, and the archive is
gzip-tested before extraction, since a proxy can return 200 with an
error page.

Verified on an arm64 rig: v0.53.1 resolved, 1/1 downloaded, the binary
runs, and the plugin's own detection finds it at
bin/pixlet/pixlet-linux-arm64.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(pixlet): anchor the version check, and don't echo response bytes raw

Both CodeRabbit findings were valid.

The shape check accepted partial matches, so "v0.53garbage", "0.53" and
"v0.5" passed it and built a download URL for a release that cannot exist
-- the failure the check was added to stop, just one step later. Anchored
at both ends now. Every tronbyt/pixlet release to date is vX.Y.Z (all 38
verified against the API), with an optional suffix left for a future -rc.1
or +build tag.

The invalid-response diagnostic printed bytes straight from whatever
answered the request. NUL and newline were filtered but escape, carriage
return and backspace were not, so an error page could rewrite the output
or bury it in a CI log. Non-printable bytes are stripped and it goes
through printf. CodeRabbit suggested hex-encoding the lot; printable
characters are kept instead, because "<!DOCTYPE html>" is the diagnostic
-- hex would make the line safe and useless.

Also corrected the comment above the parse. It asserted GitHub returns
this JSON on a single line; the API is pretty-printed by default, and I
could not get a single-line response from two machines across five header
variants. The single-line case is real (it is what produces
"mentions_count", and the failing device's error named
pixlet_mentions_count_linux-arm64.tar.gz), but it is a shape to be robust
against, not a constant. As written the comment invites the next reader to
check by hand, see pretty JSON, and conclude the fix was unnecessary.

Tests drive the real script with a stubbed curl: the tag resolves from
both response shapes, non-release values fall back, an HTTP error is
reported as a download failure rather than surfacing later as a tar error,
a non-archive body is rejected before extraction, and the diagnostic
cannot carry control bytes. The stub honours -f the way real curl does --
without that, the HTTP-error test passed against the old script too, since
both end at 0/1 and only the reporting layer differs.

Mutation-checked: 10 of the 16 fail against the pre-fix script, and the 5
covering these two findings fail against this branch's previous state.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:35:02 -04:00
0901d044d3 fix(plugins): report updates that completed, not ones that were queued (#460)
run_scheduled_updates_with_changes() snapshotted plugin_last_update,
called run_scheduled_updates(), and diffed the two to answer "whose data
just changed".

But run_scheduled_updates() only enqueues. The work runs on the update
worker and stamps plugin_last_update there, after this method has already
returned, so the two snapshots were always identical and the result was
always an empty list. The only path that ever worked was the synchronous
kill-switch, where update() runs inline.

Vegas is the caller. That empty list is what feeds mark_plugin_updated(),
which drops the cached content for a plugin whose data moved -- so a
segment kept scrolling whatever it was first built from. It is the
failure the coordinator's own comments describe: last night's live game
still drawn as live the next morning. On a live rig: zero update ticks in
twenty minutes, with weather, stocks and news all updating on schedule.

The worker now records each completed update in a ledger and the call
drains it, reporting what has finished since the previous poll rather
than what this call enqueued. That costs one tick of latency -- Vegas
polls every ~4s -- and is correct whichever side of the queue the work
lands on. Failure paths are excluded: they stamp the timestamp too, to
space out retries, but no fresh data exists.

Verified on the rig it was found on: 0 update ticks before, 208 in
twenty-five minutes after, naming real plugins.

The behavioural tests here would pass with both production call sites
deleted, which mutation testing caught -- they drive the ledger directly.
So there is also a structural test asserting the invariant at the source:
wherever a successful update stamps plugin_last_update, it must record
the completion. Writing it immediately caught that _record_update_failure
stamps the same field and must not be included.

Mutation-checked: removing either call site, removing both, and dropping
the drain's clear are all caught.


Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:28:40 -04:00
08265c1135 feat(vegas): let live content keep its place in the ticker (#457)
* feat(vegas): let live content keep its place in the ticker

Live content used to preempt Vegas outright: while any plugin reported
live priority the display controller refused to run the ticker at all and
showed a full-screen scoreboard instead. Keeping the marquee meant not
seeing live scores; seeing live scores meant losing the marquee.

Two changes, both off by default.

vegas_scroll.live_in_ticker keeps the ticker running through a live game.
Three places assumed the takeover and all three now honour it: the
controller's gate, the coordinator's per-frame pause, and the rotation
switch that would otherwise move current_mode_index underneath a ticker
that never yields.

And the rotation is no longer a strict round robin. It was one slot per
plugin per cycle, so with a dozen plugins enabled a live score came round
once a lap and could be minutes old on screen. A plugin can now hold
several slots, placed by Smooth Weighted Round-Robin -- the same
scheduler the sports plugins already use to rotate their own games. The
property that matters is that repeats are spread through the cycle
rather than clumped: three in a row and then silence would be worse than
no boost at all.

Weight comes from the plugin first, via a new optional
get_vegas_priority_weight(), then from the core: live content earns
live_weight, everything else 1. So existing plugins gain the behaviour
without changes, and the hook exists for the one thing the core cannot
work out -- the core can see that a game is live but not whose, so only
the plugin can say a favorite is playing.

Documented in ADVANCED_FEATURES (worked example, why weights are per
plugin not per game, and that frequency is not freshness),
CONFIG_REFERENCE, PLUGIN_API_REFERENCE, and the config template.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(vegas): carry the new keys through config, and correct two docs

Three findings from CodeRabbit, all valid.

to_dict() and update() enumerate keys explicitly and had not learned the
three new ones, so get_status() never reported them and a live config
change never applied -- turning live_in_ticker on in the web UI would
have done nothing until a restart. update() clamps the weights exactly
as from_config does.

The vegas_scroll key count in ADVANCED_FEATURES said 29; the template
has 30. My arithmetic, not the reviewer's.

The third was a documentation error rather than a code one, and I have
fixed it the other way round. The docs claimed a raising
get_vegas_priority_weight() is treated as weight 1. The code instead
falls through to the core's own live-content check, and that is the
better behaviour: the hook is only how a plugin asks for *more* than
live_weight, and has_live_priority/has_live_content are separate methods
guarded separately, so a plugin with a broken weight calculation should
lose the favorite distinction and keep the live boost. Said so in the
code, the base-plugin docstring and the API reference.

The test fake now fails in each place independently, because the two
failures mean different things: a broken hook still earns live_weight, a
plugin that cannot say whether it is live has nothing to fall back on
and weighs 1.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ui/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(vegas): stop the heaviest plugin doubling across the cycle seam

Smooth Weighted Round-Robin spaces repeats well within a pass, but it
schedules the heaviest item first and usually last as well. The strip
loops, so those two are neighbours: the marquee showed the same plugin
twice running at exactly the one join a within-cycle check cannot see.
Observed on a live rig at 28 slots -- gaps of 6, 7, 7, 7 and then 1.

Rotating the list does not fix it. Rotation preserves the cyclic order
exactly, so it moves where the seam is drawn rather than the adjacency
itself; the trailing entry has to be swapped with one from the middle.

The first version swapped with the first slot that merely fitted, which
undid the spacing this exists to protect -- it moved a repeat from a gap
of 7 into a gap of 2, more clumped than the seam had ever been. It now
picks the candidate furthest from any other appearance, so the repeat
lands in the widest gap.

Left alone when no candidate exists. A plugin holding most of the slots
has to neighbour itself, and scheduling it is better than refusing to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(vegas): stop the seam repair creating the duplicate it removes

Swapping the trailing repeat with a middle slot moves two elements, and
the candidate filter only guarded one of them. It checked the neighbours
`repeated` would acquire at j, but not what the displaced element would
sit beside at the end -- so ['a','b','c','d','x','y','x','a'] came back
as [...,'x','x'], the seam duplicate traded for a fresh one. Reported by
CodeRabbit with that exact case.

Adding the missing condition fixed it and immediately broke something
else: schedule[j] is schedule[-2] when j is the second-to-last slot, so
that candidate was always excluded, and ['a','b','c','a'] lost the only
repair it has. The same class of mistake twice, from reasoning about
which neighbours two moved elements end up with.

So it no longer reasons. It performs each candidate swap, counts the
cyclic duplicates in the result, and keeps the best one that has none --
preferring whichever leaves the boosted plugin most evenly spread. When
no such swap exists the schedule is returned untouched, which is the
unavoidable case: a plugin holding most of the slots has to neighbour
itself.

Fuzzed across 6,956 seam schedules: none made worse, none lost an entry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 17:02:36 -04:00
5713fd20a7 fix(calendar): implement the OAuth and calendar-listing endpoints (#458)
* fix(calendar): implement the OAuth and calendar-listing endpoints

The plugin's config advertised a three-step setup and only step 1
existed. Step 3's picker fetched
/api/v3/plugins/calendar/list-calendars, which was never registered, so
Flask fell through to the global 404 handler and the user saw "Resource
not found" -- a message that names nothing and points nowhere. Step 2
had no endpoint at all, so even a working picker would have found no
token to list with.

Two routes, following the pattern the spotify and ytm plugins already
use for their own auth scripts:

  POST /plugins/calendar/authenticate    two-step Google OAuth
  GET  /plugins/calendar/list-calendars  calendars for the picker

The authenticate route drives calendar_registration.py, which the plugin
already ships and which was written expressly for this -- it reads a
redirect URL on stdin and prints one JSON object. It takes two calls
because a human has to visit Google in between; the script persists the
PKCE verifier from the first call for the second, without which the
exchange fails with "Missing code verifier".

The listing route reads the token directly rather than shelling out
again: the picker is interactive and a subprocess per click is slower
than the API call it would wrap. It refreshes an expired token in place,
sorts the primary calendar first, and drops entries with no id, which
could not be selected anyway.

Both name the plugin when it is not installed, rather than reproducing
the anonymous 404 that started this.

Verified against the live Google API on the dev rig: HTTP 200 with the
account's real calendars.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(calendar): one input for the auth code, and a louder warning

Two things reported after testing the flow.

There were two boxes and no way to tell which to use. The config
template's string branch dispatches widgets from an allow-list of names,
and anything missing from it falls through to a plain input type=text --
so the field rendered both the widget's own box and a stray one for the
same key. google-oauth is now on that list, which is all the widget ever
needed to render in place of the fallback rather than beside it.

And the warning that the redirect page fails to load was small grey text
under a link, which is where it is least likely to be read. It is now an
amber callout that leads with "The next page will fail to load. That is
expected." The failure lands at exactly the moment the user has to act
on it, and it looks precisely like the flow breaking rather than
working. The paste box is labelled too, rather than relying on a
placeholder that vanishes on focus.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(calendar): redact script diagnostics, and page the calendar list

Three findings from CodeRabbit, all valid.

Raw subprocess output was being returned to the client -- the script's
stderr on one path, and its own error payload on another. CodeQL flagged
the same line. That script handles OAuth client secrets and interpolates
exceptions into its messages, so either could carry a secret or a path.
Both now go to the log unredacted, where they are worth having in full,
and reach the client through a redactor.

That redactor already existed inside describe_exception, which only
takes exceptions. Split out as redact_text: an exception is not the only
thing worth returning, and a subprocess's stderr is just as capable of
quoting a token.

calendarList.list returns 100 entries per page by default, caps at 250,
and hands back a nextPageToken when there are more. Reading one page
would have hidden calendars from the picker with nothing to say the list
was cut short. It now pages, asking for 250 at a time, bounded at ten
pages so a malformed token cannot spin.

And a test helper was a lambda where ruff wants a def.

The five new tests fail against the previous commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(calendar): redact the last two raw exception interpolations

CodeQL flagged four exposure paths. Two were mine and genuinely raw: the
OSError from failing to spawn calendar_registration.py, which carries the
interpreter path and whatever the OS chose to say, and the ImportError
for the Google libraries, whose message named the missing module by
interpolating the exception directly. Both now go through
describe_exception, and the unredacted text goes to the log.

The other two are the repo-wide pattern from PR #448 -- 67 handlers on
main already return details=describe_exception(e), and these two new
handlers follow it. That function is the sanitizer: it strips URL
userinfo, auth headers and credential-shaped key=value pairs, collapses
to one line and caps the length. CodeQL's taint tracking cannot see a
sanitizer it has no model for, so it reports the flow regardless.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(calendar): announce status changes, name the paste box, drop a no-op

Three more from the review, all valid.

The status line is written after every async call -- the consent link is
ready, the exchange failed -- and was a plain paragraph, so a screen
reader was told none of it. It is a live region now.

The paste box had a visible label that was never associated with it, so
its only accessible name was the placeholder, which disappears on focus:
precisely when the value is being pasted. The label now points at the
input by id.

And a conditional in the test helper returned the same value from both
branches, which Ruff flags as RUF034. It was left over from making the
fake page; one page is all those cases need, and TestPagination builds
its own sequences.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* test(calendar): assert the accessibility relationships, not their parts

The previous assertions searched for role="status", aria-live, a label
`for` and an input `id` independently, so they passed whether or not
those belonged together. Two attributes on different elements announce
nothing, and a `for` that names something other than the input leaves it
just as anonymous.

Both attributes are now asserted on the status element itself, and the
label and input are checked to go through the same identifier rather
than merely both existing. Verified by mutation: a mismatched pair and a
displaced aria-live are both caught.

Reported by CodeRabbit, against tests I had written two commits earlier.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 15:25:43 -04:00
23 changed files with 1999 additions and 33 deletions
+3
View File
@@ -130,6 +130,9 @@
"plugin_rotation_order": [],
"use_short_date_format": true,
"vegas_scroll": {
"live_in_ticker": false,
"live_weight": 3,
"favorite_live_weight": 5,
"enabled": false,
"scroll_speed": 50,
"separator_width": 32,
+89 -1
View File
@@ -64,10 +64,98 @@ JSON is optional.
| `target_fps` | `125` | Target frame rate |
| `buffer_ahead` | `2` | Number of plugins buffered ahead |
This table is a subset — `display.vegas_scroll` supports 26 keys in
This table is a subset — `display.vegas_scroll` supports 30 keys in
total. See the full list in
[CONFIG_REFERENCE.md](CONFIG_REFERENCE.md#displayvegas_scroll--continuous-scroll-mode).
### Live Content in the Ticker
By default, live content **preempts** Vegas mode: while any plugin reports
live priority, the display controller refuses to run the ticker and shows
that plugin's full-screen display instead. You get a big readable scoreboard,
but the marquee stops entirely for the duration of the game.
Set `live_in_ticker` to keep the ticker running and let live content take
**extra turns inside it** instead:
```json
"vegas_scroll": {
"live_in_ticker": true,
"live_weight": 3,
"favorite_live_weight": 5
}
```
#### Why weights exist
The rotation is otherwise a strict round robin — every plugin appears exactly
once per cycle. With a dozen plugins enabled, a live score comes round once a
lap and can be minutes old by the time you see it. A weight of *N* gives a
plugin *N* slots per cycle.
The slots are placed by **Smooth Weighted Round-Robin**, the same scheduler
the sports plugins use internally to rotate their own games. The important
property is that repeats are *spread through the cycle* rather than clumped:
three appearances in a row followed by a long silence would be worse than not
boosting at all.
Twelve plugins, with a favorite's baseball game and an ordinary live hockey
game (`live_weight: 3`, `favorite_live_weight: 5`):
```
baseball > hockey > weather > clock > baseball
stocks > news > flights > baseball > hockey
calendar > f1 > music > baseball > tides
birds > hockey > baseball
```
18 slots for 12 plugins. Baseball appears 5 times, hockey 3, everything else
once, and no plugin ever appears twice in a row — **including across the seam**
where the cycle loops back on itself. Smooth Weighted Round-Robin schedules the
heaviest item first and usually last as well, so the strip would otherwise show
it twice running at exactly the one join a within-cycle check cannot see. The
trailing repeat is moved into the widest remaining gap. Where a double is
unavoidable — a plugin holding most of the slots has to neighbour itself — the
schedule is left as it is.
#### Where the weight comes from
For each plugin in the rotation, in order:
1. **The plugin's own answer.** If it implements
`get_vegas_priority_weight()` and returns a number, that wins. This is the
only route for favorite-team awareness — the core can see *that* a game is
live, but not *whose*, so a scoreboard has to say so itself.
2. **The core's default.** When the plugin returns `None` (the base-class
default), a plugin where both `has_live_priority()` and `has_live_content()`
are true gets `live_weight`.
3. **Everything else** gets 1.
Because of step 2, **existing plugins need no changes** — any scoreboard with
`live_priority` enabled already gets extra turns. Step 1 is opt-in, for
plugins that want to distinguish a favorite's game from any other live game.
Weights are clamped to 110. A weight of 1 is no boost; a weight below 1 would
drop the plugin from the rotation entirely, which is never what is meant.
#### Things worth knowing
- **Weights are per plugin, not per game.** A scoreboard showing four live
games still occupies one slot at a time, rotating its own games within that
slot using its own `favorite_live_boost`. This controls how often the
*plugin* comes round.
- **The ticker is zero-sum.** Giving baseball 5 slots does not make the cycle
faster; it makes the cycle *longer* and everything else proportionally
rarer. If you want live scores sooner in wall-clock terms, pair this with a
smaller `plugins_per_cycle`.
- **Frequency is not freshness.** Each appearance redraws from the plugin's
current data (`refresh_updated_plugins()` drops cached content when a
plugin's data changes), but how current that data is depends on the
plugin's own `live_update_interval`. Showing a stale score five times a lap
is no better than showing it once.
- **Everything still appears.** A boost never starves another plugin out of
the cycle; low-weight plugins keep their single slot.
### Per-Plugin Configuration
Override Vegas behavior for specific plugins:
+5 -1
View File
@@ -104,7 +104,8 @@ logical image to multiple chained physical panels.
## `display.vegas_scroll` — continuous scroll mode
Read by `src/vegas_mode/config.py` (`VegasScrollConfig.from_config`). See
[ADVANCED_FEATURES.md](ADVANCED_FEATURES.md) for behavior details.
[ADVANCED_FEATURES.md](ADVANCED_FEATURES.md) for behavior details, including
[live content in the ticker](ADVANCED_FEATURES.md#live-content-in-the-ticker).
| Key | Type / default |
|---|---|
@@ -135,6 +136,9 @@ Read by `src/vegas_mode/config.py` (`VegasScrollConfig.from_config`). See
| `max_cycle_duration` | int, `240` |
| `frame_based_scrolling` | bool, `true` — frame-count-based scroll stepping |
| `scroll_delay` | float, `0.02` — seconds between scroll updates (~50 FPS) |
| `live_in_ticker` | bool, `false` — keep scrolling during live games instead of handing the display to a full-screen scoreboard |
| `live_weight` | int, `3` (110) — slots per cycle for a plugin with live content |
| `favorite_live_weight` | int, `5` (110) — slots per cycle when a plugin reports a favorite team is live |
## `sync` — multi-display synchronization
+41
View File
@@ -170,6 +170,47 @@ Default returns `False`.
List of display modes to show during a live takeover. Default returns the
plugin's `display_modes` from its manifest.
#### `get_vegas_priority_weight() -> Optional[int]`
How many slots per Vegas cycle this plugin should get. Default returns
`None`, which defers to the core.
The Vegas ticker is otherwise a strict round robin — every plugin appears
exactly once per cycle — so with a dozen plugins enabled a live score can be
minutes stale by the time it comes round. A weight of *N* gives the plugin
*N* slots per cycle, spread evenly through it rather than clumped.
**You usually do not need this.** When the hook returns `None`, the core
already gives a plugin `vegas_scroll.live_weight` whenever
`has_live_priority()` and `has_live_content()` are both true. Live sports get
extra turns with no code at all.
Implement it only when the plugin knows something the core cannot. The
motivating case is favorite teams — the core can see *that* a game is live,
but not *whose*:
```python
def get_vegas_priority_weight(self):
if not (self.has_live_priority() and self.has_live_content()):
return None # let the core decide
vegas = self.global_config.get('display', {}).get('vegas_scroll', {})
if self._favorite_is_live():
return vegas.get('favorite_live_weight', 5)
return vegas.get('live_weight', 3)
```
The weight is per *plugin*, not per game: a scoreboard showing four live games
still occupies one slot at a time and rotates its own games within it. Values
are clamped to 110 by the caller. An exception here is caught and logged, and
the core then falls back to its own live-content check — so a plugin whose
weight calculation is broken still gets `live_weight` for a game that really
is live, rather than being demoted to 1.
Only consulted when the user has set `vegas_scroll.live_in_ticker`. With the
default (`false`) live content preempts Vegas entirely and there is no ticker
to be weighted within. See
[ADVANCED_FEATURES.md](ADVANCED_FEATURES.md#live-content-in-the-ticker).
### Vegas scroll hooks
Vegas mode shows multiple plugins as a single continuous scroll instead of
+43 -5
View File
@@ -24,9 +24,29 @@ echo "========================================"
# Auto-detect latest version if needed
if [ "$PIXLET_VERSION" = "latest" ]; then
echo "Detecting latest version..."
PIXLET_VERSION=$(curl -s "https://api.github.com/repos/${REPO}/releases/latest" | grep '"tag_name"' | sed -E 's/.*"([^"]+)".*/\1/')
if [ -z "$PIXLET_VERSION" ]; then
echo "Failed to detect latest version, using fallback"
# When this response arrives on a single line -- as it did on the device
# where Starlark apps were failing -- `grep '"tag_name"'` matches the whole
# document and a greedy `sed 's/.*"([^"]+)".*/\1/'` captures the LAST
# quoted token in it rather than the tag. That resolved to
# "mentions_count", which built a download URL for a release that does not
# exist. (The API is pretty-printed by default, which is why the old
# command looks correct when you try it by hand -- but the formatting is
# not something to depend on.) Match the field itself and take the value
# after it, which is right for either shape.
PIXLET_VERSION=$(curl -fsSL "https://api.github.com/repos/${REPO}/releases/latest" \
| grep -o '"tag_name"[[:space:]]*:[[:space:]]*"[^"]*"' \
| head -n1 \
| sed -E 's/.*:[[:space:]]*"([^"]*)".*/\1/')
# A wrong-but-non-empty value is what made the old bug silent, so check the
# shape rather than just that something came back. Anchored at both ends: a
# partial match would accept "v0.53garbage" or "0.53" and build a URL for a
# release that cannot exist, which is the failure this check is here to
# stop. Every tronbyt/pixlet release to date is vX.Y.Z; the optional suffix
# leaves room for a future -rc.1 or +build tag.
if ! printf '%s' "$PIXLET_VERSION" \
| grep -qE '^v[0-9]+\.[0-9]+\.[0-9]+([-+][0-9A-Za-z.-]+)?$'; then
echo "Could not detect the latest version (got: '${PIXLET_VERSION:-<empty>}'), using fallback"
PIXLET_VERSION="v0.50.2"
fi
fi
@@ -67,8 +87,26 @@ download_binary() {
temp_dir=$(mktemp -d -p "$PROJECT_ROOT" -t pixlet_download.XXXXXXXXXX)
local temp_file="$temp_dir/$archive_name"
if ! curl -L -o "$temp_file" "$url" 2>/dev/null; then
echo "✗ Failed to download $arch"
# -f so an HTTP error is a failure. Without it curl writes the 404 body
# to the file and exits 0, and the first sign of trouble is tar saying
# "not in gzip format" about what is actually a page of HTML.
if ! curl -fL -o "$temp_file" "$url" 2>/dev/null; then
echo "✗ Failed to download $arch from $url"
rm -rf "$temp_dir"
return 1
fi
# Belt and braces: a mirror or proxy can return 200 with an error page.
if ! gzip -t "$temp_file" 2>/dev/null; then
echo "✗ Downloaded file is not a gzip archive: $url"
# These bytes come from whatever answered the request, so strip
# everything non-printable before echoing them: an error page carrying
# terminal escapes would otherwise be able to rewrite this output or
# bury it in a CI log. Printable characters are kept rather than
# hex-encoding the lot, because "<!DOCTYPE html>" is the diagnostic.
local first_bytes
first_bytes=$(head -c 60 "$temp_file" | tr -cd '[:print:]')
printf ' (first bytes: %s)\n' "$first_bytes"
rm -rf "$temp_dir"
return 1
fi
+18 -2
View File
@@ -1694,6 +1694,12 @@ class DisplayController:
logger.warning("Error checking live priority for %s: %s", mode_name, e)
return live
def _vegas_keeps_live_in_ticker(self) -> bool:
"""Whether live content should stay in the ticker instead of preempting it."""
coordinator = getattr(self, 'vegas_coordinator', None)
config = getattr(coordinator, 'vegas_config', None)
return bool(getattr(config, 'live_in_ticker', False))
def _check_live_priority(self, advance=False):
"""Return the live-priority mode to display, or None if nothing is live.
@@ -1907,14 +1913,24 @@ class DisplayController:
# Check for live priority content and switch to it immediately.
# advance=True so multiple simultaneously-live games take turns
# (round-robin) instead of pinning to the first plugin.
if not self.on_demand_active and not wifi_status_data:
# Skipped when the ticker is keeping live content: switching
# the rotation underneath Vegas would move current_mode_index
# and stash a resume point for a takeover that never happens.
if (not self.on_demand_active and not wifi_status_data
and not (self._is_vegas_mode_active()
and self._vegas_keeps_live_in_ticker())):
live_priority_mode = self._check_live_priority(advance=True)
self._apply_live_priority(live_priority_mode)
# Vegas scroll mode - continuous ticker across all plugins
# Priority: on-demand > wifi-status > live-priority > vegas > normal rotation
if self._is_vegas_mode_active() and not wifi_status_data:
live_mode = self._check_live_priority()
# Live content normally preempts the ticker entirely. With
# vegas_scroll.live_in_ticker the marquee keeps running and
# the live plugin takes extra turns inside it instead --
# see StreamManager._apply_priority_weights.
live_mode = (None if self._vegas_keeps_live_in_ticker()
else self._check_live_priority())
if not live_mode:
try:
# Run Vegas mode iteration
+42
View File
@@ -555,6 +555,48 @@ class BasePlugin(ABC):
"""
return False
def get_vegas_priority_weight(self) -> Optional[int]:
"""How many slots per Vegas cycle this plugin should get, or None.
The Vegas ticker is otherwise a strict round robin: every plugin
appears exactly once per cycle. With a dozen plugins enabled that puts
minutes between a live score and its next appearance. A weight of N
gives the plugin N slots per cycle, spread evenly through it rather
than clumped together.
Return ``None`` (the default) to let the core decide. It gives a
plugin ``vegas_scroll.live_weight`` when ``has_live_priority()`` and
``has_live_content()`` are both true, and 1 otherwise -- so live sports
already get extra turns without implementing this at all.
Implement it only when the plugin knows something the core cannot. The
motivating case is favorite teams: the core can see *that* a game is
live but not *whose*, so a scoreboard that wants its favorite's game
shown more often than other live games has to say so::
def get_vegas_priority_weight(self):
if not (self.has_live_priority() and self.has_live_content()):
return None # let the core decide
cfg = self.global_config.get('display', {}).get('vegas_scroll', {})
if self._favorite_is_live():
return cfg.get('favorite_live_weight', 5)
return cfg.get('live_weight', 3)
The weight is per *plugin*, not per game. A scoreboard showing four
live games still occupies one slot at a time and rotates its own games
within that slot; this controls how often the plugin itself comes
round.
Raising is safe: the core logs it and falls back to its own
live-content check, so a broken weight calculation costs the plugin
the favorite distinction but not the live boost.
Returns:
Slots per cycle (clamped to 1..10 by the caller), or None to
defer to the core's own live-content weighting.
"""
return None
def get_live_modes(self) -> List[str]:
"""
Get list of display modes that should be used during live priority takeover.
+12 -1
View File
@@ -64,8 +64,19 @@ class PluginHealthTracker:
cache_key, max_age=None, memory_ttl=0 if force_reload else None
)
if cached:
if isinstance(cached, dict) and cached:
return cached
# A cache entry that is not a dict means the persisted state was written
# by something other than _save_health_state (a key collision, a partial
# write, a restored backup). Returning it verbatim makes every caller
# blow up on .get(), which takes the display down in a restart loop that
# survives reboots because the bad entry is on disk. Discard and rebuild.
if cached is not None and not isinstance(cached, dict):
self.logger.warning(
f"Discarding malformed health state for {plugin_id}: expected "
f"dict, got {type(cached).__name__}. Falling back to defaults."
)
# Default state
return {
+41 -18
View File
@@ -125,6 +125,14 @@ class PluginManager:
self._plugin_locks: Dict[str, threading.Lock] = {}
self._plugin_locks_guard = threading.Lock()
self._update_worker: Optional[threading.Thread] = None
# Plugin ids whose update() has finished since the last time anyone
# asked. Updates are dispatched to a worker thread, so a caller that
# wants to know "whose data just changed" cannot learn it by diffing
# plugin_last_update around run_scheduled_updates() -- that call only
# enqueues, and the timestamp is stamped later, on the worker. See
# run_scheduled_updates_with_changes().
self._completed_updates: set = set()
self._completed_updates_lock = threading.Lock()
self._synchronous_updates = False
if self.config_manager is not None:
try:
@@ -1025,6 +1033,7 @@ class PluginManager:
if success:
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = scheduled_time
self._note_update_completed(plugin_id)
self.state_manager.record_update(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
if self.health_tracker:
@@ -1089,28 +1098,41 @@ class PluginManager:
def run_scheduled_updates_with_changes(self, current_time: Optional[float] = None) -> List[str]:
"""
Like run_scheduled_updates(), but also returns the plugin_ids whose
plugin_last_update timestamp actually advanced during this call.
Like run_scheduled_updates(), but also reports which plugins have
fresh data -- the ids whose update() has finished since the last
call, not necessarily the ones enqueued by this one.
The before/after snapshots and the update pass itself are each
individually lock-protected against concurrent plugin_last_update
mutation (Vegas mode calls this from its own background
update-tick thread, racing the main render loop's plugin updates),
so callers get an atomic "who got fresh data" answer without
reaching into plugin_last_update themselves. The lock is not held
across the update pass so slow/blocking plugin update() calls don't
serialize against other plugin_last_update readers.
That distinction is the whole point. This used to snapshot
plugin_last_update, call run_scheduled_updates(), and diff. But
run_scheduled_updates() only *enqueues*: the work runs on the
update worker and the timestamp is stamped there, after this method
has already returned. The two snapshots were therefore always
identical and the result was always empty, so Vegas never learned
that any plugin's data had changed and kept scrolling whatever a
segment was first built from -- last night's live game still drawn
as live the next morning. The only path that ever worked was the
synchronous kill-switch, where update() runs inline.
Reporting completions instead of enqueues costs a poll's worth of
latency (the Vegas tick runs every ~4s) and is correct regardless of
which side of the queue the work lands on.
"""
with self._plugin_last_update_lock:
old_times = dict(self.plugin_last_update)
self.run_scheduled_updates(current_time)
return self.drain_completed_updates()
with self._plugin_last_update_lock:
return [
plugin_id for plugin_id, new_time in self.plugin_last_update.items()
if new_time > old_times.get(plugin_id, 0.0)
]
def _note_update_completed(self, plugin_id: str) -> None:
"""Record that a plugin's update() finished, for the next poll."""
with self._completed_updates_lock:
self._completed_updates.add(plugin_id)
def drain_completed_updates(self) -> List[str]:
"""Return and clear the plugin ids whose update() has since finished."""
with self._completed_updates_lock:
if not self._completed_updates:
return []
done = sorted(self._completed_updates)
self._completed_updates.clear()
return done
def update_all_plugins(self) -> None:
"""
@@ -1135,6 +1157,7 @@ class PluginManager:
if success:
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = time.time()
self._note_update_completed(plugin_id)
self.state_manager.record_update(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
else:
+44
View File
@@ -125,6 +125,32 @@ class VegasModeConfig:
plugin_order: List[str] = field(default_factory=list)
excluded_plugins: Set[str] = field(default_factory=set)
# --- Live content in the ticker -------------------------------------
#
# By default a live game preempts Vegas entirely: the display controller
# refuses to run the ticker while any plugin reports live priority, and you
# get the full-screen scoreboard instead. Set live_in_ticker to keep the
# marquee running and let live content take extra turns within it.
#
# The rotation is otherwise a strict round robin -- every plugin appears
# exactly once per cycle -- so with a dozen plugins enabled a live score
# comes round once a lap and can be minutes old on screen. Weighting lets a
# plugin claim several slots per cycle instead.
#
# Weights are per plugin, not per game: a scoreboard showing four live
# games still occupies one slot at a time, and rotates its own games within
# that slot using its own favorite_live_boost.
live_in_ticker: bool = False
# Slots per cycle for a plugin reporting live content. 1 disables the boost
# and restores the plain round robin.
live_weight: int = 3
# Slots per cycle for a plugin whose live content involves a favorite team.
# Only plugins implementing get_vegas_priority_weight() can claim this --
# the core cannot tell whose game is on, so the plugin reports it.
favorite_live_weight: int = 5
# Performance settings
target_fps: int = 125 # Target frame rate
buffer_ahead: int = 2 # Number of plugins to buffer ahead
@@ -175,6 +201,12 @@ class VegasModeConfig:
overflow_mode=str(vegas_config.get('overflow_mode', 'rotate')),
plugin_order=list(vegas_config.get('plugin_order', [])),
excluded_plugins=set(vegas_config.get('excluded_plugins', [])),
live_in_ticker=bool(vegas_config.get('live_in_ticker', False)),
# Clamped: a weight below 1 would drop the plugin from the rotation
# entirely, and a very large one starves everything else.
live_weight=max(1, min(10, int(vegas_config.get('live_weight', 3)))),
favorite_live_weight=max(
1, min(10, int(vegas_config.get('favorite_live_weight', 5)))),
target_fps=int(vegas_config.get('target_fps', 125)),
buffer_ahead=int(vegas_config.get('buffer_ahead', 2)),
frame_based_scrolling=vegas_config.get('frame_based_scrolling', True),
@@ -204,6 +236,9 @@ class VegasModeConfig:
'lead_in_width': self.lead_in_width,
'plugins_per_cycle': self.plugins_per_cycle,
'max_plugin_width_ratio': self.max_plugin_width_ratio,
'live_in_ticker': self.live_in_ticker,
'live_weight': self.live_weight,
'favorite_live_weight': self.favorite_live_weight,
'overflow_mode': self.overflow_mode,
'plugin_order': self.plugin_order,
'excluded_plugins': list(self.excluded_plugins),
@@ -371,6 +406,15 @@ class VegasModeConfig:
if 'enabled' in vegas_config:
self.enabled = vegas_config['enabled']
if 'live_in_ticker' in vegas_config:
self.live_in_ticker = bool(vegas_config['live_in_ticker'])
# Clamped exactly as from_config does: a weight below 1 would drop the
# plugin from the rotation, and a huge one starves everything else.
if 'live_weight' in vegas_config:
self.live_weight = max(1, min(10, int(vegas_config['live_weight'])))
if 'favorite_live_weight' in vegas_config:
self.favorite_live_weight = max(
1, min(10, int(vegas_config['favorite_live_weight'])))
if 'scroll_speed' in vegas_config:
self.scroll_speed = float(vegas_config['scroll_speed'])
if 'separator_width' in vegas_config:
+6
View File
@@ -528,6 +528,12 @@ class VegasModeCoordinator:
if not self._live_priority_check:
return False
if self.vegas_config.live_in_ticker:
# The ticker keeps live content rather than yielding to it; the
# extra turns are arranged in the rotation itself, so there is
# nothing to pause for.
return False
try:
live_mode = self._live_priority_check()
if live_mode:
+139
View File
@@ -406,6 +406,8 @@ class StreamManager:
)
logger.info("Ordered plugins: %s", ordered_plugins)
ordered_plugins = self._apply_priority_weights(ordered_plugins)
# Atomically update shared state under lock to avoid races with prefetchers
with self._buffer_lock:
self._ordered_plugins = ordered_plugins
@@ -417,6 +419,143 @@ class StreamManager:
logger.info("=" * 60)
def _plugin_weight(self, plugin_id: str) -> int:
"""Slots per cycle for one plugin.
A plugin may answer for itself via get_vegas_priority_weight() -- the
only way favorite-team awareness can reach here, since the core can see
that a game is live but not whose. When it declines (returns None, the
default), live content earns ``live_weight`` and everything else 1.
"""
plugin = None
try:
plugin = self.plugin_manager.plugins.get(plugin_id)
except (AttributeError, TypeError):
return 1
if plugin is None:
return 1
try:
if hasattr(plugin, 'get_vegas_priority_weight'):
declared = plugin.get_vegas_priority_weight()
if declared is not None:
return max(1, min(10, int(declared)))
except Exception:
# Deliberately falls through to the core's own live check rather
# than demoting to 1. The plugin's weight calculation is broken,
# but has_live_priority() and has_live_content() are separate
# methods guarded separately below -- a plugin that genuinely has
# a live game should still get live_weight for it.
logger.exception("[%s] get_vegas_priority_weight() failed", plugin_id)
try:
if (hasattr(plugin, 'has_live_priority')
and hasattr(plugin, 'has_live_content')
and plugin.has_live_priority()
and plugin.has_live_content()):
return self.config.live_weight
except Exception:
logger.exception("[%s] live-content check failed", plugin_id)
return 1
def _apply_priority_weights(self, ordered: List[str]) -> List[str]:
"""Expand the rotation so weighted plugins take several turns per cycle.
Smooth Weighted Round-Robin, the same scheduler the sports plugins use
to rotate their own games: a plugin of weight N appears N times per
cycle, and the repeats are spaced through the cycle rather than
clumped, so a live score is never three-in-a-row followed by a long
silence.
Returns the input unchanged when nothing is weighted, which is both the
common case and the pre-existing behaviour.
"""
if not ordered or not self.config.live_in_ticker:
return ordered
weights = {pid: self._plugin_weight(pid) for pid in ordered}
total = sum(weights.values())
if total <= len(ordered):
return ordered # nothing boosted; plain round robin
current = {pid: 0 for pid in ordered}
schedule: List[str] = []
for _ in range(total):
for pid in ordered:
current[pid] += weights[pid]
picked = max(current, key=lambda p: current[p])
current[picked] -= total
schedule.append(picked)
schedule = self._unclump_seam(schedule)
boosted = {p: w for p, w in weights.items() if w > 1}
logger.info(
"Vegas rotation weighted: %d slots for %d plugins (boosted: %s)",
len(schedule), len(ordered), boosted)
return schedule
@staticmethod
def _unclump_seam(schedule: List[str]) -> List[str]:
"""Stop the heaviest plugin sitting on both ends of the cycle.
Smooth Weighted Round-Robin spaces repeats well *within* a pass, but
it schedules the heaviest item first and often last too. The strip
loops, so those two are neighbours: the one place the marquee shows
the same plugin twice running is the seam between cycles.
Rotating the list cannot fix this. Rotation preserves the cyclic order
exactly, so it only moves where the seam is drawn, not the adjacency
itself. The trailing entry has to be swapped with one from the middle
whose neighbours differ from it, which breaks the pair without
creating another.
Left alone when no such position exists -- a rotation short enough or
lopsided enough to have none is one where the plugin is unavoidably
adjacent to itself anyway.
"""
if len(schedule) < 3 or schedule[0] != schedule[-1]:
return schedule
repeated = schedule[-1]
size = len(schedule)
def cyclic_doubles(seq) -> int:
return sum(1 for i in range(size) if seq[i] == seq[(i + 1) % size])
def clearance(seq, value) -> int:
"""Smallest cyclic gap between appearances of `value`."""
at = [i for i, v in enumerate(seq) if v == value]
if len(at) < 2:
return size
return min(min((b - a) % size, (a - b) % size)
for i, a in enumerate(at) for b in at[i + 1:])
# Try each swap and judge the result, rather than reasoning about which
# neighbours the two moved elements will end up with. That reasoning is
# where the first version went wrong: it guarded the slot `repeated`
# moves into but not the one the displaced element lands in, so
# ['a','b','c','d','x','y','x','a'] came back ending ['x','x'] -- the
# seam duplicate traded for a fresh one.
best = None
best_clearance = -1
for j in range(1, size - 1):
candidate = list(schedule)
candidate[j], candidate[-1] = candidate[-1], candidate[j]
if cyclic_doubles(candidate):
continue
# Among the repairs that work, prefer the one that leaves the
# boosted plugin most evenly spread; taking the first that merely
# fits moved a repeat from a gap of 7 into a gap of 2.
spread = clearance(candidate, repeated)
if spread > best_clearance:
best, best_clearance = candidate, spread
# None exists when the value is unavoidably adjacent to itself -- a
# plugin holding most of the slots has to be. Schedule it as it is
# rather than refuse.
return best if best is not None else schedule
def _prefetch_content(self, count: int = 1) -> None:
"""
Prefetch content for upcoming plugins.
+19
View File
@@ -77,6 +77,25 @@ def describe_exception(exc: BaseException,
"""
message = str(exc).strip()
text = f"{type(exc).__name__}: {message}" if message else type(exc).__name__
return redact_text(text, max_length)
def redact_text(text: str, max_length: int = _MAX_DETAIL_LENGTH) -> str:
"""Make arbitrary text safe to hand back over HTTP.
Split out of describe_exception because exceptions are not the only thing
worth returning: a subprocess's stderr, or a message a helper script
printed, is just as useful to a user and just as capable of carrying a
token or a password in it.
Args:
text: The text to redact
max_length: Truncate beyond this many characters
Returns:
A single line, credentials replaced, length capped.
"""
text = text or ''
# Order matters: the URL and header forms are more specific than the
# generic key=value pattern, which would otherwise chew the scheme.
text = _REDACT_URL_USERINFO.sub(r'\1<redacted>\3', text)
+2 -2
View File
@@ -10,8 +10,8 @@ WorkingDirectory=__PROJECT_ROOT_DIR__
ExecStart=/usr/bin/python3 __PROJECT_ROOT_DIR__/scripts/utils/wifi_monitor_daemon.py --interval 30
Restart=on-failure
RestartSec=10
StandardOutput=syslog
StandardError=syslog
StandardOutput=journal
StandardError=journal
SyslogIdentifier=ledmatrix-wifi-monitor
[Install]
+5 -1
View File
@@ -9,7 +9,11 @@ User=root
WorkingDirectory=__PROJECT_ROOT_DIR__
Environment=PYTHONDONTWRITEBYTECODE=1
ExecStart=/usr/bin/python3 __PROJECT_ROOT_DIR__/run.py
Restart=on-failure
# Restart=always, not on-failure: run.py exiting 0 (a clean shutdown path taken
# for a reason that no longer applies, e.g. a config reload) would otherwise leave
# the service stopped and the panel dark indefinitely, with systemd considering
# that a successful outcome and never bringing it back.
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
+180
View File
@@ -0,0 +1,180 @@
"""
Tests for scripts/download_pixlet.sh -- release-tag resolution and download guards.
Background: Starlark apps render through the pixlet binary, and the installer
that fetches it failed silently. It resolved the release tag by grepping the
GitHub API response for '"tag_name"' and taking the last quoted token on the
match with a greedy sed. When the response arrives on one line that token is
"mentions_count", not the tag, so the script built a URL for a release that
cannot exist -- and `curl -L -o` without -f wrote the 404 body to the file and
exited 0, so the first sign of trouble was tar reporting "not in gzip format"
about a page of HTML.
The API is pretty-printed by default, which is exactly why this needs a test:
by hand the old command looks correct, and the failure only appears when the
formatting changes. These drive the real script with a stubbed curl on PATH, so
both response shapes are covered without touching the network.
"""
import re
import shutil
import subprocess
from pathlib import Path
import pytest
SCRIPT = Path(__file__).resolve().parent.parent / "scripts" / "download_pixlet.sh"
PRETTY = """{
"url": "https://api.github.com/repos/tronbyt/pixlet/releases/12345",
"id": 12345,
"tag_name": "v0.53.1",
"name": "v0.53.1",
"draft": false,
"prerelease": false,
"mentions_count": 3
}
"""
# The shape that broke it: one line, and the last quoted token is not the tag.
MINIFIED = (
'{"url":"https://api.github.com/repos/tronbyt/pixlet/releases/12345",'
'"id":12345,"tag_name":"v0.53.1","name":"v0.53.1","draft":false,'
'"prerelease":false,"mentions_count":3}'
)
def run_script(tmp_path, api_body, download=None):
"""Run the real script against a stubbed curl.
Args:
api_body: what the stub returns for the api.github.com request.
download: bytes to write for a release-asset request, or None to make
that request fail the way `curl -f` does on an HTTP error.
"""
root = tmp_path / "project"
(root / "scripts").mkdir(parents=True)
shutil.copy(SCRIPT, root / "scripts" / "download_pixlet.sh")
api_file = tmp_path / "api.json"
api_file.write_text(api_body)
stub_dir = tmp_path / "stub"
stub_dir.mkdir()
asset_file = tmp_path / "asset.bin"
if download is not None:
asset_file.write_bytes(download)
# Stands in for curl, including the -f semantics the fix turns on: without
# -f, real curl writes the error body to the output file and exits 0, which
# is what let a 404 masquerade as a successful download. The stub has to
# honour that or a test of the fix would pass against the old script too.
(stub_dir / "curl").write_text(f"""#!/bin/bash
out=""
url=""
fail_on_error=0
while [ $# -gt 0 ]; do
case "$1" in
-o) out="$2"; shift 2 ;;
-*f*) fail_on_error=1; shift ;;
-*) shift ;;
*) url="$1"; shift ;;
esac
done
if [[ "$url" == *api.github.com* ]]; then
cat {api_file}
exit 0
fi
if [ -f "{asset_file}" ]; then
cp "{asset_file}" "$out"
exit 0
fi
# No asset: stand in for an HTTP 404.
if [ "$fail_on_error" = "1" ]; then
exit 22
fi
printf '<!DOCTYPE html><html>404 Not Found</html>' > "$out"
exit 0
""")
(stub_dir / "curl").chmod(0o755)
return subprocess.run(
["bash", str(root / "scripts" / "download_pixlet.sh")],
capture_output=True, text=True,
env={"PATH": f"{stub_dir}:/usr/bin:/bin:/usr/sbin:/sbin",
"PIXLET_VERSION": "latest"},
)
def resolved_version(result):
match = re.search(r"^Version: (.+)$", result.stdout, re.M)
assert match, f"no version line in output:\n{result.stdout}"
return match.group(1).strip()
def test_script_is_syntactically_valid():
result = subprocess.run(["bash", "-n", str(SCRIPT)], capture_output=True, text=True)
assert result.returncode == 0, result.stderr
@pytest.mark.parametrize("body,label", [(PRETTY, "pretty"), (MINIFIED, "minified")])
def test_tag_is_resolved_from_either_response_shape(tmp_path, body, label):
"""The minified case is the regression: the last quoted token there is
"mentions_count", which is what the old greedy sed captured."""
result = run_script(tmp_path, body)
assert resolved_version(result) == "v0.53.1", f"{label}: {result.stdout}"
assert "mentions_count" not in result.stdout
@pytest.mark.parametrize(
"tag",
["mentions_count", "v0.53garbage", "0.53", "v0.5", "", "v0.53.1 ; echo pwned"],
)
def test_a_tag_that_is_not_a_release_falls_back(tmp_path, tag):
"""A wrong-but-non-empty value is what made the original bug silent, so the
check is on the shape. Partial matches must not pass: "v0.53garbage" and
"0.53" would build a URL for a release that cannot exist."""
result = run_script(tmp_path, '{"tag_name": "%s"}' % tag)
assert resolved_version(result) == "v0.50.2", result.stdout
assert "using fallback" in result.stdout
@pytest.mark.parametrize("tag", ["v0.53.1", "v1.0.0", "v0.54.0-rc.1", "v1.2.3+build.4"])
def test_real_release_tag_shapes_are_accepted(tmp_path, tag):
assert resolved_version(run_script(tmp_path, '{"tag_name": "%s"}' % tag)) == tag
def test_an_http_error_is_reported_as_a_failed_download(tmp_path):
"""Without curl -f the 404 body lands in the file and curl exits 0, so the
failure surfaced two steps later as tar complaining about gzip -- about
what was really a page of HTML. It has to be reported where it happened.
Both versions end at 0/1, so asserting only on the count would pass against
the old script; the discriminating part is which layer reports it.
"""
result = run_script(tmp_path, PRETTY, download=None)
assert "Download complete: 0/1 succeeded" in result.stdout
assert "✓ Downloaded" not in result.stdout
assert "Failed to download" in result.stdout
assert "Failed to extract" not in result.stdout, (
"an HTTP error should not surface as an extraction failure")
def test_a_non_archive_response_is_rejected_before_extraction(tmp_path):
result = run_script(tmp_path, PRETTY, download=b"<!DOCTYPE html><html>502 Bad Gateway")
assert "not a gzip archive" in result.stdout
assert "Download complete: 0/1 succeeded" in result.stdout
def test_the_diagnostic_cannot_smuggle_terminal_escapes(tmp_path):
"""Those bytes come from whatever answered the request. An error page
carrying escapes must not be able to rewrite the output or bury it."""
hostile = b"<!DOCTYPE html>\x1b[2J\x1b[31mgone\x1b[0m\rHTTP 200 OK\x08\x08"
result = run_script(tmp_path, PRETTY, download=hostile)
assert "not a gzip archive" in result.stdout
printed = re.search(r"^\s*\(first bytes: (.*)\)$", result.stdout, re.M)
assert printed, f"no diagnostic line:\n{result.stdout}"
assert "DOCTYPE" in printed.group(1), "the useful part of the page was dropped"
for forbidden in ("\x1b", "\r", "\x08", "\x00"):
assert forbidden not in printed.group(1), (
f"control byte {forbidden!r} reached the terminal")
+182
View File
@@ -0,0 +1,182 @@
#!/usr/bin/env python3
"""
Tests that "which plugins have fresh data" survives the async update worker.
Regression under test: run_scheduled_updates_with_changes() snapshotted
plugin_last_update, called run_scheduled_updates(), and diffed the two. But
run_scheduled_updates() only *enqueues* -- the work runs on the update worker
and stamps the timestamp there, after the method has already returned. The
snapshots were therefore always identical and the result always empty.
Vegas depends on that result: it is what calls mark_plugin_updated(), which
drops the cached content for a plugin whose data changed. With it always
empty, a segment kept scrolling whatever it was first built from -- the
"last night's live game still drawn as live the next morning" failure the
coordinator comments describe. Observed on a live rig: zero update ticks in
twenty minutes, with weather, stocks and news all updating.
Run: python -m pytest test/test_update_change_reporting.py -v
"""
import ast
import inspect
import sys
import threading
import unittest
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.plugin_system.plugin_manager import PluginManager # noqa: E402
def _manager():
"""A PluginManager with only the update-reporting state initialised."""
manager = PluginManager.__new__(PluginManager)
manager._completed_updates = set()
manager._completed_updates_lock = threading.Lock()
return manager
class DrainCompletedUpdates(unittest.TestCase):
def setUp(self):
self.manager = _manager()
def test_nothing_completed_reports_nothing(self):
self.assertEqual(self.manager.drain_completed_updates(), [])
def test_a_completed_update_is_reported(self):
self.manager._note_update_completed("news")
self.assertEqual(self.manager.drain_completed_updates(), ["news"])
def test_draining_clears_so_the_next_poll_is_empty(self):
self.manager._note_update_completed("news")
self.manager.drain_completed_updates()
self.assertEqual(
self.manager.drain_completed_updates(), [],
"a plugin must be reported once per update, not on every poll, "
"or Vegas would drop its cached content every few seconds")
def test_repeated_completions_between_polls_collapse(self):
for _ in range(5):
self.manager._note_update_completed("weather")
self.assertEqual(self.manager.drain_completed_updates(), ["weather"])
def test_multiple_plugins_are_all_reported(self):
for plugin_id in ("news", "weather", "ledmatrix-stocks"):
self.manager._note_update_completed(plugin_id)
self.assertEqual(self.manager.drain_completed_updates(),
["ledmatrix-stocks", "news", "weather"])
class CompletionReportingIsAsyncSafe(unittest.TestCase):
"""The point of the change: completion may land after the call returns."""
def setUp(self):
self.manager = _manager()
def test_an_update_completing_after_the_call_is_still_reported(self):
"""The exact shape of the bug.
The enqueueing call sees nothing, because the worker has not run yet.
The next poll must report it -- under the old diff it was lost, since
the second snapshot was taken before the worker ever stamped.
"""
first = self.manager.drain_completed_updates()
self.assertEqual(first, [], "nothing has finished yet")
# The worker finishes some time later, on its own thread.
worker = threading.Thread(
target=self.manager._note_update_completed, args=("news",))
worker.start()
worker.join()
self.assertEqual(
self.manager.drain_completed_updates(), ["news"],
"an update that finishes between polls must still be reported")
def test_concurrent_completions_are_not_lost(self):
ids = ["plugin-%02d" % i for i in range(40)]
threads = [threading.Thread(target=self.manager._note_update_completed,
args=(pid,)) for pid in ids]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
self.assertEqual(self.manager.drain_completed_updates(), sorted(ids))
def test_a_completion_during_a_drain_is_not_swallowed(self):
"""A drain must not clear an entry it did not report."""
self.manager._note_update_completed("news")
reported = self.manager.drain_completed_updates()
# ...worker finishes another one immediately afterwards
self.manager._note_update_completed("weather")
self.assertEqual(reported, ["news"])
self.assertEqual(self.manager.drain_completed_updates(), ["weather"])
class EveryStampRecordsACompletion(unittest.TestCase):
"""The ledger is only correct if the production paths actually fill it.
Asserting on the mechanics alone passes even when nothing calls
_note_update_completed -- verified by deleting the call sites, which the
behavioural tests above did not notice. This checks the invariant at the
source: wherever a successful update stamps plugin_last_update, it must
also record the completion, or Vegas silently stops being told.
"""
def test_success_paths_record_the_completion(self):
import src.plugin_system.plugin_manager as pm
tree = ast.parse(inspect.getsource(pm))
stamps = []
for node in ast.walk(tree):
if not isinstance(node, ast.With):
continue
# `with self._plugin_last_update_lock:` blocks that stamp a real
# time on success. Two stamps are deliberately excluded: the 0.0
# written at registration, and the failure path, which backs the
# timestamp off to space out retries -- neither means fresh data.
assigns_time = any(
isinstance(stmt, ast.Assign)
and any(isinstance(t, ast.Subscript)
and getattr(t.value, "attr", None) == "plugin_last_update"
for t in stmt.targets)
and not (isinstance(stmt.value, ast.Constant)
and stmt.value.value == 0.0)
and "failure" not in ast.dump(stmt.value)
for stmt in node.body
)
if assigns_time:
stamps.append(node)
self.assertGreaterEqual(
len(stamps), 2,
"expected the worker and inline success paths to stamp the time; "
"if this drops, the search below is looking at the wrong thing")
for stamp in stamps:
enclosing = self._enclosing_function(tree, stamp)
calls = [n for n in ast.walk(enclosing)
if isinstance(n, ast.Call)
and getattr(n.func, "attr", None) == "_note_update_completed"]
self.assertTrue(
calls,
"%s stamps plugin_last_update on success but never calls "
"_note_update_completed, so a plugin's fresh data would never "
"be reported and Vegas would keep its stale cached content"
% enclosing.name)
@staticmethod
def _enclosing_function(tree, target):
best = None
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
if node.lineno <= target.lineno <= (node.end_lineno or node.lineno):
if best is None or node.lineno > best.lineno:
best = node
return best
if __name__ == "__main__":
unittest.main(verbosity=2)
+300
View File
@@ -0,0 +1,300 @@
"""Tests that live content can take extra turns inside the Vegas ticker.
Vegas was a strict round robin -- every plugin exactly once per cycle -- and
live content did not appear in it at all, because the display controller
refused to run the ticker while anything was live. With a dozen plugins
enabled that left a live score either absent or minutes stale.
Two things change, both off by default. `live_in_ticker` keeps the marquee
running instead of yielding to a full-screen takeover, and the rotation is
expanded by Smooth Weighted Round-Robin so a weighted plugin gets several
slots per cycle, spaced through it rather than clumped.
Weights are per plugin, not per game: a scoreboard showing four live games
still occupies one slot at a time and rotates its own games within it.
"""
from unittest.mock import Mock
import pytest
from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.stream_manager import StreamManager
class FakePlugin:
"""A plugin that can fail in each place independently.
hook_raises and live_raises are separate because they mean different
things: a broken weight calculation should still leave the core's own
live-content check usable, while a plugin that cannot answer whether it is
live at all has nothing left to fall back on.
"""
def __init__(self, live=False, declared=None, raises=False,
hook_raises=False, live_raises=False):
self._live = live
self._declared = declared
self._hook_raises = hook_raises or raises
self._live_raises = live_raises or raises
self.enabled = True
def has_live_priority(self):
if self._live_raises:
raise RuntimeError("cannot say whether I am live")
return self._live
def has_live_content(self):
return self._live
def get_vegas_priority_weight(self):
if self._hook_raises:
raise RuntimeError("weight calculation blew up")
return self._declared
def _manager(plugins, **cfg):
config = VegasModeConfig(live_in_ticker=cfg.pop('live_in_ticker', True), **cfg)
pm = Mock()
pm.plugins = plugins
sm = StreamManager.__new__(StreamManager)
sm.config = config
sm.plugin_manager = pm
return sm
def _counts(schedule):
return {p: schedule.count(p) for p in set(schedule)}
def _max_gap(schedule, plugin_id):
"""Largest gap between consecutive appearances, wrapping around."""
at = [i for i, p in enumerate(schedule) if p == plugin_id]
if len(at) < 2:
return len(schedule)
gaps = [b - a for a, b in zip(at, at[1:])]
gaps.append(len(schedule) - at[-1] + at[0])
return max(gaps)
class TestWeightsComeFromTheRightPlace:
def test_a_quiet_plugin_gets_one_slot(self):
sm = _manager({'clock': FakePlugin()})
assert sm._plugin_weight('clock') == 1
def test_live_content_earns_the_configured_weight(self):
sm = _manager({'mlb': FakePlugin(live=True)}, live_weight=4)
assert sm._plugin_weight('mlb') == 4
def test_a_plugin_may_answer_for_itself(self):
# The only route for favorite-team awareness: the core can see that a
# game is live, not whose.
sm = _manager({'mlb': FakePlugin(live=True, declared=7)}, live_weight=3)
assert sm._plugin_weight('mlb') == 7
def test_declaring_none_defers_to_the_core(self):
sm = _manager({'mlb': FakePlugin(live=True, declared=None)}, live_weight=3)
assert sm._plugin_weight('mlb') == 3
def test_a_declared_weight_is_clamped(self):
sm = _manager({'a': FakePlugin(declared=99), 'b': FakePlugin(declared=0)})
assert sm._plugin_weight('a') == 10
assert sm._plugin_weight('b') == 1
def test_a_plugin_that_raises_everywhere_weighs_one(self):
sm = _manager({'bad': FakePlugin(raises=True)})
assert sm._plugin_weight('bad') == 1
def test_a_broken_hook_still_earns_the_live_boost(self):
# The hook is only how a plugin asks for *more* than live_weight.
# Losing it should cost the favorite distinction, not the live boost:
# has_live_priority/has_live_content are separate and still work.
sm = _manager({'mlb': FakePlugin(live=True, hook_raises=True)},
live_weight=4)
assert sm._plugin_weight('mlb') == 4
def test_a_broken_hook_on_a_quiet_plugin_weighs_one(self):
sm = _manager({'clock': FakePlugin(live=False, hook_raises=True)},
live_weight=4)
assert sm._plugin_weight('clock') == 1
def test_a_plugin_that_cannot_say_whether_it_is_live_weighs_one(self):
# Nothing left to fall back on, so no boost.
sm = _manager({'mlb': FakePlugin(live=True, live_raises=True)},
live_weight=4)
assert sm._plugin_weight('mlb') == 1
def test_an_unknown_plugin_weighs_one(self):
assert _manager({})._plugin_weight('ghost') == 1
class TestTheSchedule:
def test_nothing_weighted_leaves_the_order_untouched(self):
order = ['weather', 'clock', 'news']
sm = _manager({p: FakePlugin() for p in order})
assert sm._apply_priority_weights(order) == order
def test_off_by_default_the_order_is_untouched(self):
order = ['weather', 'mlb', 'news']
sm = _manager({'weather': FakePlugin(), 'mlb': FakePlugin(live=True),
'news': FakePlugin()}, live_in_ticker=False, live_weight=3)
assert sm._apply_priority_weights(order) == order
def test_a_live_plugin_takes_its_share_of_slots(self):
order = ['weather', 'mlb', 'news', 'clock']
sm = _manager({'weather': FakePlugin(), 'mlb': FakePlugin(live=True),
'news': FakePlugin(), 'clock': FakePlugin()},
live_weight=3)
schedule = sm._apply_priority_weights(order)
counts = _counts(schedule)
assert counts['mlb'] == 3, counts
assert counts['weather'] == counts['news'] == counts['clock'] == 1, counts
assert len(schedule) == 6
def test_every_plugin_still_appears(self):
# A boost must not starve anything out of the cycle.
order = ['a', 'b', 'c', 'd', 'e', 'f']
plugins = {p: FakePlugin() for p in order}
plugins['a'] = FakePlugin(live=True, declared=10)
sm = _manager(plugins)
schedule = sm._apply_priority_weights(order)
assert set(schedule) == set(order), set(order) - set(schedule)
def test_nothing_doubles_across_the_cycle_seam(self):
# The strip loops, so the last slot neighbours the first. Smooth
# Weighted Round-Robin schedules the heaviest item first and often
# last too, which put the one clump the algorithm exists to avoid at
# the one place a within-cycle check cannot see.
order = ['baseball', 'weather', 'geochron', 'flights', 'stocks',
'oftheday', 'youtube', 'stocknews', 'leaderboard',
'countdown', 'odds', 'f1', 'football', 'music']
plugins = {p: FakePlugin() for p in order}
plugins['baseball'] = FakePlugin(live=True, declared=5)
plugins['football'] = FakePlugin(live=True, declared=3)
schedule = _manager(plugins)._apply_priority_weights(order)
n = len(schedule)
doubles = [schedule[i] for i in range(n)
if schedule[i] == schedule[(i + 1) % n]]
assert not doubles, "%r repeats across the seam in %r" % (doubles, schedule)
def test_the_seam_repair_keeps_every_slot(self):
order = ['a', 'b', 'c', 'd', 'e', 'f']
plugins = {p: FakePlugin() for p in order}
plugins['a'] = FakePlugin(live=True, declared=4)
schedule = _manager(plugins)._apply_priority_weights(order)
assert _counts(schedule)['a'] == 4, _counts(schedule)
assert sorted(schedule) == sorted(
['a'] * 4 + ['b', 'c', 'd', 'e', 'f']), schedule
def test_the_repair_uses_the_widest_gap(self):
# Moving the trailing repeat into the first slot that merely fits
# undoes the spacing: on a 28-slot rotation that turned a gap of 7
# into a gap of 2, which is more clumped than the seam ever was.
order = ['a'] + ['p%d' % i for i in range(13)]
plugins = {p: FakePlugin() for p in order}
plugins['a'] = FakePlugin(live=True, declared=4)
schedule = _manager(plugins)._apply_priority_weights(order)
at = [i for i, p in enumerate(schedule) if p == 'a']
gaps = [b - a for a, b in zip(at, at[1:])]
gaps.append(len(schedule) - at[-1] + at[0])
ideal = len(schedule) / len(at)
assert min(gaps) >= ideal / 2, "gaps %r for ideal %.1f" % (gaps, ideal)
def test_an_unavoidable_double_is_left_alone(self):
# Five of seven slots are the same plugin, so it must neighbour
# itself. Better to schedule it than to refuse or loop forever.
order = ['a', 'b', 'c']
plugins = {p: FakePlugin() for p in order}
plugins['a'] = FakePlugin(live=True, declared=5)
schedule = _manager(plugins)._apply_priority_weights(order)
assert _counts(schedule) == {'a': 5, 'b': 1, 'c': 1}, _counts(schedule)
assert set(schedule) == {'a', 'b', 'c'}
def test_the_repair_never_creates_a_new_double(self):
# The first version guarded the slot the repeated value moves *into*
# but not the one the displaced element lands in, so this traded the
# seam duplicate for a fresh one and came back ending ['x', 'x'].
sm = _manager({})
out = sm._unclump_seam(['a', 'b', 'c', 'd', 'x', 'y', 'x', 'a'])
n = len(out)
doubles = [out[i] for i in range(n) if out[i] == out[(i + 1) % n]]
assert not doubles, "%r in %r" % (doubles, out)
assert sorted(out) == sorted(['a', 'b', 'c', 'd', 'x', 'y', 'x', 'a'])
def test_the_last_two_slots_are_a_usable_swap(self):
# Reasoning about indices said this candidate was unsafe because
# schedule[j] is schedule[-2]; after the swap its neighbour is the
# repeated value, not itself. Refusing it left the only repair this
# schedule has on the table.
assert _manager({})._unclump_seam(['a', 'b', 'c', 'a']) == ['a', 'b', 'a', 'c']
def test_no_seam_schedule_is_ever_made_worse(self):
import random
sm = _manager({})
random.seed(11)
checked = 0
for size in range(3, 10):
for _ in range(400):
original = [random.choice('abcd') for _ in range(size)]
if original[0] != original[-1]:
continue
checked += 1
out = sm._unclump_seam(list(original))
n = len(out)
before = sum(1 for i in range(n)
if original[i] == original[(i + 1) % n])
after = sum(1 for i in range(n) if out[i] == out[(i + 1) % n])
assert after <= before, (original, out)
assert sorted(out) == sorted(original), (original, out)
assert checked > 100, "the generator stopped producing seam cases"
def test_a_schedule_too_short_to_repair_is_returned_as_is(self):
sm = _manager({})
assert sm._unclump_seam(['a', 'a']) == ['a', 'a']
assert sm._unclump_seam(['a']) == ['a']
assert sm._unclump_seam([]) == []
def test_a_schedule_with_no_seam_clash_is_untouched(self):
sm = _manager({})
plain = ['a', 'b', 'c', 'a', 'd']
assert sm._unclump_seam(plain) == plain
def test_repeats_are_spread_not_clumped(self):
# The point of Smooth Weighted Round-Robin. Three-in-a-row followed by
# a long silence would be worse than not boosting at all.
order = ['weather', 'mlb', 'news', 'clock', 'stocks', 'f1']
plugins = {p: FakePlugin() for p in order}
plugins['mlb'] = FakePlugin(live=True)
sm = _manager(plugins, live_weight=3)
schedule = sm._apply_priority_weights(order)
assert _counts(schedule)['mlb'] == 3
# Evenly spread over 8 slots means a gap of about 3, never 6.
assert _max_gap(schedule, 'mlb') <= 4, schedule
# And never twice running.
assert not any(a == b == 'mlb' for a, b in zip(schedule, schedule[1:])), schedule
def test_a_favorite_outranks_another_live_game(self):
order = ['weather', 'mlb', 'nhl']
sm = _manager({'weather': FakePlugin(),
'mlb': FakePlugin(live=True, declared=5),
'nhl': FakePlugin(live=True)}, live_weight=2)
counts = _counts(sm._apply_priority_weights(order))
assert counts['mlb'] == 5 and counts['nhl'] == 2 and counts['weather'] == 1, counts
def test_an_empty_rotation_is_harmless(self):
assert _manager({})._apply_priority_weights([]) == []
class TestConfigParsing:
def test_defaults_preserve_todays_behaviour(self):
cfg = VegasModeConfig.from_config({})
assert cfg.live_in_ticker is False
assert cfg.live_weight == 3 and cfg.favorite_live_weight == 5
@pytest.mark.parametrize("given,expected", [(0, 1), (-4, 1), (99, 10), (4, 4)])
def test_weights_are_clamped(self, given, expected):
cfg = VegasModeConfig.from_config(
{'display': {'vegas_scroll': {'live_weight': given}}})
assert cfg.live_weight == expected
@@ -0,0 +1,408 @@
"""Tests the calendar plugin's OAuth and calendar-listing endpoints.
The plugin's config UI advertised a three-step setup, but only step 1 existed
on the server. Step 3's picker fetched /api/v3/plugins/calendar/list-calendars,
which was never registered, so Flask fell through to the global 404 handler and
the user saw "Resource not found" with nothing to say which resource. Step 2
had no endpoint either, and no field in the schema at all, even though the
plugin ships calendar_registration.py written expressly for a web-driven
two-step flow.
These cover the two new routes: that they exist, that they fail with something
actionable rather than a bare 404, and that the shapes the widgets consume are
what the server actually sends.
"""
import json
import pickle
import sys
from pathlib import Path
import pytest
project_root = Path(__file__).parent.parent.parent
sys.path.insert(0, str(project_root))
from web_interface.blueprints import api_v3 as mod # noqa: E402
@pytest.fixture
def client(monkeypatch, tmp_path):
"""A test client whose calendar plugin lives in tmp_path."""
from flask import Flask
plugin_dir = tmp_path / 'calendar'
plugin_dir.mkdir()
app = Flask(__name__)
app.register_blueprint(mod.api_v3, url_prefix='/api/v3')
app.config['TESTING'] = True
monkeypatch.setattr(mod, '_calendar_plugin_dir', lambda: plugin_dir)
with app.test_client() as c:
c.plugin_dir = plugin_dir
yield c
@pytest.fixture
def uninstalled(monkeypatch):
from flask import Flask
app = Flask(__name__)
app.register_blueprint(mod.api_v3, url_prefix='/api/v3')
app.config['TESTING'] = True
monkeypatch.setattr(mod, '_calendar_plugin_dir', lambda: None)
with app.test_client() as c:
yield c
class TestTheRoutesExistAtAll:
"""The original bug: the URLs the widgets call were not registered."""
def test_list_calendars_is_routed(self, client):
response = client.get('/api/v3/plugins/calendar/list-calendars')
# Reaching the handler is the whole point; what it then says about
# missing setup is TestItSaysWhatIsWrong's business.
assert response.status_code != 404, "still unrouted"
assert response.get_json()['message'] != 'Resource not found'
def test_authenticate_is_routed(self, client):
response = client.post('/api/v3/plugins/calendar/authenticate', json={})
assert response.status_code != 404, "still unrouted"
assert response.get_json()['message'] != 'Resource not found'
def test_both_urls_match_what_the_widgets_request(self):
# The widgets hardcode these; a rename on either side reintroduces the
# original bug silently.
picker = Path(project_root) / 'web_interface/static/v3/js/widgets/google-calendar-picker.js'
oauth = Path(project_root) / 'web_interface/static/v3/js/widgets/google-oauth.js'
assert '/api/v3/plugins/calendar/list-calendars' in picker.read_text(encoding='utf-8')
assert '/api/v3/plugins/calendar/authenticate' in oauth.read_text(encoding='utf-8')
source = (Path(project_root) / 'web_interface/blueprints/api_v3.py').read_text(encoding='utf-8')
assert "'/plugins/calendar/list-calendars'" in source
assert "'/plugins/calendar/authenticate'" in source
def test_the_oauth_widget_is_dispatched_not_rendered_as_a_text_box(self):
# The string branch of the config template dispatches on an allow-list
# of widget names; anything missing from it silently falls through to a
# plain <input type="text">. That produced two boxes on the calendar
# page -- the widget's own, and a stray one for the same field -- and
# no way to tell which to paste into.
template = (Path(project_root)
/ 'web_interface/templates/v3/partials/plugin_config.html'
).read_text(encoding='utf-8')
allow_list_line = [ln for ln in template.splitlines()
if "str_widget in [" in ln]
assert allow_list_line, "the string widget allow-list moved"
assert "'google-oauth'" in allow_list_line[0], allow_list_line[0]
def test_the_widget_script_is_served(self):
base = (Path(project_root) / 'web_interface/templates/v3/base.html'
).read_text(encoding='utf-8')
assert 'widgets/google-oauth.js' in base
def test_the_status_line_is_announced(self):
# Every message the widget gives arrives after an async call, so a
# screen reader hears nothing unless the element is a live region.
widget = (Path(project_root)
/ 'web_interface/static/v3/js/widgets/google-oauth.js'
).read_text(encoding='utf-8')
# Both attributes must be on the *status* element. Searching for them
# separately would pass with each on a different node, which announces
# nothing.
assert "status.setAttribute('role', 'status')" in widget, widget[:0]
assert "status.setAttribute('aria-live', 'polite')" in widget
def test_the_paste_box_has_an_accessible_name(self):
# A visible label is not enough on its own: without the association the
# input's only name is a placeholder, which vanishes on focus -- which
# is exactly when the value is being pasted.
widget = (Path(project_root)
/ 'web_interface/static/v3/js/widgets/google-oauth.js'
).read_text(encoding='utf-8')
# The binding is what matters, not that both lines exist: a `for` and
# an `id` that disagree leave the input just as anonymous. Both must
# go through the same identifier.
import re as _re
for_target = _re.search(r"codeLabel\.setAttribute\('for',\s*(\w+)\)", widget)
id_source = _re.search(r"codeInput\.id\s*=\s*(\w+)", widget)
assert for_target and id_source, (for_target, id_source)
assert for_target.group(1) == id_source.group(1), (
"label points at %r but the input is %r"
% (for_target.group(1), id_source.group(1)))
def test_the_failed_page_is_called_out_loudly(self):
# The loopback redirect lands on a browser error page at exactly the
# moment the user has to act. In small grey text it gets missed and the
# flow reads as broken while it is working.
widget = (Path(project_root)
/ 'web_interface/static/v3/js/widgets/google-oauth.js'
).read_text(encoding='utf-8')
assert 'expected' in widget.lower()
assert 'amber' in widget, "the warning is not visually distinguished"
class TestItSaysWhatIsWrong:
def test_listing_without_a_token_asks_for_step_2(self, client):
response = client.get('/api/v3/plugins/calendar/list-calendars')
assert response.status_code == 400
body = response.get_json()
assert body['status'] == 'error'
assert 'step 2' in body['message'].lower(), body['message']
def test_authenticating_without_credentials_asks_for_step_1(self, client):
response = client.post('/api/v3/plugins/calendar/authenticate', json={})
assert response.status_code == 400
assert 'step 1' in response.get_json()['message'].lower()
def test_an_uninstalled_plugin_says_so(self, uninstalled):
for response in (
uninstalled.get('/api/v3/plugins/calendar/list-calendars'),
uninstalled.post('/api/v3/plugins/calendar/authenticate', json={}),
):
assert response.status_code == 404
# A 404 here is honest -- but it must name the plugin, not read as
# the generic "Resource not found" that started this.
assert 'not installed' in response.get_json()['message'].lower()
class TestTheScriptRunner:
def test_it_returns_the_json_the_script_prints(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text(
'print(\'{"status": "success", "auth_url": "https://x"}\')\n',
encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert error is None
assert payload['auth_url'] == 'https://x'
def test_it_ignores_noise_before_the_json(self, tmp_path):
# An import warning or a library writing to stdout would otherwise
# make the last-line parse fail.
script = tmp_path / 'calendar_registration.py'
script.write_text(
'print("some library warning")\n'
'print(\'{"status": "success"}\')\n', encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert error is None and payload['status'] == 'success'
def test_it_passes_stdin_through(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text(
'import sys, json\n'
'print(json.dumps({"status": "success", "got": sys.stdin.read().strip()}))\n',
encoding='utf-8')
payload, _ = mod._run_calendar_registration(tmp_path, 'http://127.0.0.1/?code=abc')
assert payload['got'] == 'http://127.0.0.1/?code=abc'
def test_a_missing_script_is_reported(self, tmp_path):
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'script not found' in error.lower()
def test_output_that_is_not_json_is_reported_with_context(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text('import sys\nsys.stderr.write("boom\\n")\n', encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'no result' in error.lower()
assert 'boom' in error
class TestListingShape:
"""The picker reads cal.id, cal.summary and cal.primary."""
def _authenticate(self, client, monkeypatch, items):
creds = type('C', (), {'expired': False, 'refresh_token': None, 'valid': True})()
(client.plugin_dir / 'token.pickle').write_bytes(pickle.dumps({'x': 1}))
monkeypatch.setattr(mod.pickle if hasattr(mod, 'pickle') else pickle,
'loads', lambda *a, **k: creds, raising=False)
import types
fake_pickle = types.SimpleNamespace(load=lambda f: creds, dump=lambda *a: None)
# Callers pass a flat list of calendars; the API returns them wrapped
# in a page. One page is all these cases need -- TestPagination builds
# its own multi-page sequences.
pages = [{'items': items}]
state = {'i': 0}
def fake_list(**kwargs):
page = pages[min(state['i'], len(pages) - 1)]
state['i'] += 1
return types.SimpleNamespace(execute=lambda: page)
def fake_build(*args, **kwargs):
return types.SimpleNamespace(
calendarList=lambda: types.SimpleNamespace(list=fake_list))
real_import = __builtins__['__import__'] if isinstance(__builtins__, dict) \
else __builtins__.__import__
def fake_import(name, *args, **kwargs):
if name == 'pickle':
return fake_pickle
if name == 'google.auth.transport.requests':
return types.SimpleNamespace(Request=object)
if name == 'googleapiclient.discovery':
return types.SimpleNamespace(build=fake_build)
return real_import(name, *args, **kwargs)
monkeypatch.setattr('builtins.__import__', fake_import)
def test_it_returns_id_summary_and_primary(self, client, monkeypatch):
self._authenticate(client, monkeypatch, [
{'id': 'b@x', 'summary': 'Work'},
{'id': 'a@x', 'summary': 'Personal', 'primary': True},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['status'] == 'success'
assert {c['id'] for c in body['calendars']} == {'a@x', 'b@x'}
assert all(set(c) == {'id', 'summary', 'primary'} for c in body['calendars'])
def test_the_primary_calendar_comes_first(self, client, monkeypatch):
# Short list, but the one the user wants is almost always their own.
self._authenticate(client, monkeypatch, [
{'id': 'z@x', 'summary': 'Aardvarks'},
{'id': 'a@x', 'summary': 'Zebras', 'primary': True},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['calendars'][0]['id'] == 'a@x'
assert body['calendars'][0]['primary'] is True
def test_a_calendar_without_a_name_still_lists(self, client, monkeypatch):
self._authenticate(client, monkeypatch, [{'id': 'noname@x'}])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['calendars'][0]['summary'] == 'noname@x'
def test_entries_without_an_id_are_dropped(self, client, monkeypatch):
# Nothing could be selected by such a row, and the checkbox value
# would be undefined.
self._authenticate(client, monkeypatch, [{'summary': 'ghost'}, {'id': 'real@x'}])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert [c['id'] for c in body['calendars']] == ['real@x']
class TestPagination:
"""calendarList.list pages at 250 and defaults to 100."""
def _paged(self, client, monkeypatch, pages):
import types
creds = type('C', (), {'expired': False, 'refresh_token': None, 'valid': True})()
(client.plugin_dir / 'token.pickle').write_bytes(b'x')
state = {'i': 0}
seen = []
def fake_list(**kwargs):
seen.append(kwargs)
page = pages[min(state['i'], len(pages) - 1)]
state['i'] += 1
return types.SimpleNamespace(execute=lambda: page)
def fake_build(*args, **kwargs):
return types.SimpleNamespace(
calendarList=lambda: types.SimpleNamespace(list=fake_list))
real_import = __builtins__['__import__'] if isinstance(__builtins__, dict) \
else __builtins__.__import__
def fake_import(name, *args, **kwargs):
if name == 'pickle':
return types.SimpleNamespace(load=lambda f: creds, dump=lambda *a: None)
if name == 'google.auth.transport.requests':
return types.SimpleNamespace(Request=object)
if name == 'googleapiclient.discovery':
return types.SimpleNamespace(build=fake_build)
return real_import(name, *args, **kwargs)
monkeypatch.setattr('builtins.__import__', fake_import)
return seen
def test_every_page_is_collected(self, client, monkeypatch):
# Taking only the first page would hide calendars from the picker with
# nothing to say the list was cut short.
self._paged(client, monkeypatch, [
{'items': [{'id': 'a@x', 'summary': 'A'}], 'nextPageToken': 't1'},
{'items': [{'id': 'b@x', 'summary': 'B'}], 'nextPageToken': 't2'},
{'items': [{'id': 'c@x', 'summary': 'C'}]},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert [c['id'] for c in body['calendars']] == ['a@x', 'b@x', 'c@x']
def test_the_page_token_is_passed_back(self, client, monkeypatch):
seen = self._paged(client, monkeypatch, [
{'items': [{'id': 'a@x', 'summary': 'A'}], 'nextPageToken': 'tok'},
{'items': [{'id': 'b@x', 'summary': 'B'}]},
])
client.get('/api/v3/plugins/calendar/list-calendars')
assert seen[0]['pageToken'] is None
assert seen[1]['pageToken'] == 'tok'
assert all(k['maxResults'] == 250 for k in seen)
def test_a_looping_token_cannot_spin_forever(self, client, monkeypatch):
# Every page claims another follows.
self._paged(client, monkeypatch, [
{'items': [{'id': 'a@x', 'summary': 'A'}], 'nextPageToken': 'same'},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['status'] == 'success'
assert len(body['calendars']) <= mod._CALENDAR_LIST_MAX_PAGES
class TestDiagnosticsAreRedacted:
def test_script_stderr_is_redacted_on_the_way_out(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text(
'import sys\n'
'sys.stderr.write("boom client_secret=hunter2 more\\n")\n',
encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'hunter2' not in error, error
assert '<redacted>' in error, error
def test_a_failing_script_payload_is_redacted(self, client):
(client.plugin_dir / 'credentials.json').write_text('{}', encoding='utf-8')
(client.plugin_dir / 'calendar_registration.py').write_text(
'import json\n'
'print(json.dumps({"status": "error", '
'"message": "Failed: client_secret=topsecret"}))\n',
encoding='utf-8')
body = client.post('/api/v3/plugins/calendar/authenticate',
json={}).get_json()
assert body['status'] == 'error'
assert 'topsecret' not in json.dumps(body), body
assert '<redacted>' in body['message'], body
def test_an_unrunnable_script_is_reported_without_raw_exception_text(self,
tmp_path,
monkeypatch):
# OSError from the spawn carries the interpreter path and whatever the
# OS chose to say; it reaches the client through the redactor like
# everything else.
script = tmp_path / 'calendar_registration.py'
script.write_text('', encoding='utf-8')
def boom(*a, **k):
raise OSError("Exec format error: token=abcd1234 /usr/bin/python3")
monkeypatch.setattr(mod.subprocess, 'run', boom)
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'abcd1234' not in error, error
assert 'OSError' in error, error
def test_a_missing_google_library_is_reported_without_raw_exception_text(
self, client, monkeypatch):
(client.plugin_dir / 'token.pickle').write_bytes(b'x')
real_import = __builtins__['__import__'] if isinstance(__builtins__, dict) \
else __builtins__.__import__
def fake_import(name, *args, **kwargs):
if name.startswith('google'):
raise ImportError("No module named 'google' password=hunter2")
return real_import(name, *args, **kwargs)
monkeypatch.setattr('builtins.__import__', fake_import)
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert 'hunter2' not in json.dumps(body), body
assert 'requirements.txt' in body['message']
+222 -1
View File
@@ -22,7 +22,7 @@ logger = logging.getLogger(__name__)
from src.web_interface.api_helpers import success_response, error_response, validate_request_json
from src.web_interface.errors import ErrorCode
from src.web_interface.secret_helpers import find_secret_fields, separate_secrets
from src.web_interface.error_handler import describe_exception
from src.web_interface.error_handler import describe_exception, redact_text
from src.plugin_system.operation_types import OperationType
from src.web_interface.validators import (
validate_file_upload
@@ -7317,6 +7317,227 @@ def upload_calendar_credentials():
logger.error('Error in upload_calendar_credentials', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
# calendarList.list pages at 250 entries maximum. Ten pages is far past any
# real account and exists only so a malformed nextPageToken cannot spin here.
_CALENDAR_LIST_MAX_PAGES = 10
def _calendar_plugin_dir() -> Optional[Path]:
"""Where the calendar plugin is installed, or None if it is not."""
if api_v3.plugin_manager:
plugin_dir = api_v3.plugin_manager.get_plugin_directory('calendar')
else:
plugin_dir = PROJECT_ROOT / 'plugins' / 'calendar'
if not plugin_dir:
return None
plugin_dir = Path(plugin_dir)
return plugin_dir if plugin_dir.exists() else None
def _run_calendar_registration(plugin_dir: Path, stdin_payload: str):
"""Run the plugin's OAuth script and return the JSON object it prints.
The script decides between web and terminal mode by whether stdin is a
tty, so it must be given a pipe. It emits one JSON object on stdout; the
last parsable line is taken, because an import warning or a library's
stderr redirection can land in front of it.
Returns (payload, error_message). Exactly one is None.
"""
script = plugin_dir / 'calendar_registration.py'
if not script.exists():
return None, 'Authentication script not found in the calendar plugin'
try:
result = subprocess.run( # nosec B603 - fixed script path inside the plugin dir
[sys.executable, str(script)],
input=stdin_payload,
capture_output=True,
text=True,
timeout=120,
cwd=str(plugin_dir),
)
except subprocess.TimeoutExpired:
return None, 'Authentication timed out after 120s'
except OSError as e:
logger.error('Could not run calendar_registration.py', exc_info=True)
return None, 'Could not run the authentication script: %s' % describe_exception(e)
for line in reversed((result.stdout or '').splitlines()):
line = line.strip()
if not line:
continue
try:
payload = json.loads(line)
except json.JSONDecodeError:
continue
if isinstance(payload, dict):
return payload, None
raw = (result.stderr or result.stdout or '').strip()
# The unredacted text goes to the log, where it is worth having in full.
# What comes back over HTTP is redacted: this is a script that handles
# OAuth client secrets, and its stderr can quote them.
if raw:
logger.error('calendar_registration.py failed (exit %s): %s',
result.returncode, raw)
return None, 'Authentication script produced no result%s' % (
': %s' % redact_text(raw) if raw else '')
@api_v3.route('/plugins/calendar/authenticate', methods=['POST'])
def authenticate_calendar():
"""Google OAuth for the calendar plugin, in the two steps it requires.
Step 1 (no body) returns the consent URL to open. Step 2 posts back the
URL Google redirected to -- it fails to load, because the redirect points
at a loopback address nothing is listening on, but the address bar carries
the authorization code -- and the script exchanges it for a token.
Two calls rather than one because the user has to visit Google in between.
The script persists the PKCE verifier from step 1 for step 2 to reuse; the
exchange fails with "Missing code verifier" otherwise.
"""
try:
plugin_dir = _calendar_plugin_dir()
if plugin_dir is None:
return jsonify({
'status': 'error',
'message': 'The calendar plugin is not installed'
}), 404
if not (plugin_dir / 'credentials.json').exists():
return jsonify({
'status': 'error',
'message': ('No credentials.json yet. Upload your Google OAuth '
'client file first (Step 1).')
}), 400
data = request.get_json(silent=True) or {}
redirect_url = (data.get('redirect_url') or data.get('code') or '').strip()
payload, error = _run_calendar_registration(plugin_dir, redirect_url)
if error:
return jsonify({'status': 'error', 'message': error}), 500
if payload.get('status') != 'success':
# The script's own diagnosis is more useful than anything that
# could be reconstructed here -- but it interpolates exceptions
# into its messages, so it reaches the client redacted and the
# original goes to the log.
logger.error('calendar authentication failed: %s', payload)
safe = dict(payload)
safe['message'] = redact_text(str(payload.get('message', '')
or 'Authentication failed'))
return jsonify(safe), 400
return jsonify(payload)
except Exception as e:
logger.error('Error in authenticate_calendar', exc_info=True)
return jsonify({'status': 'error',
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)}), 500
@api_v3.route('/plugins/calendar/list-calendars', methods=['GET'])
def list_calendar_calendars():
"""The calendars this account can see, for the config picker.
Reads the token the OAuth flow wrote rather than shelling out again: the
picker is used interactively and a subprocess per click is slower than the
API call it would be wrapping.
"""
try:
plugin_dir = _calendar_plugin_dir()
if plugin_dir is None:
return jsonify({
'status': 'error',
'message': 'The calendar plugin is not installed'
}), 404
token_file = plugin_dir / 'token.pickle'
if not token_file.exists():
return jsonify({
'status': 'error',
'message': ('Not authenticated with Google yet. Complete Step 2 '
'first, then load your calendars.')
}), 400
try:
import pickle
from google.auth.transport.requests import Request as GoogleRequest
from googleapiclient.discovery import build as build_google_service
except ImportError as e:
return jsonify({
'status': 'error',
# The name of the missing module is the whole diagnosis, but it
# arrives as an exception, so it goes through the redactor like
# any other -- an ImportError can quote a path.
'message': ('The Google API libraries are not installed. Install '
"the calendar plugin's requirements.txt. (%s)"
% describe_exception(e))
}), 500
with open(token_file, 'rb') as handle:
# Written only by this plugin's own OAuth flow, into its own
# directory, and read here exactly as the plugin itself reads it.
creds = pickle.load(handle) # nosec B301 - locally generated token
if creds and creds.expired and creds.refresh_token:
creds.refresh(GoogleRequest())
with open(token_file, 'wb') as handle:
pickle.dump(creds, handle)
os.chmod(token_file, 0o600)
if not creds or not creds.valid:
return jsonify({
'status': 'error',
'message': ('Stored Google credentials are no longer valid. '
'Run Step 2 again to re-authenticate.')
}), 400
service = build_google_service('calendar', 'v3', credentials=creds)
# calendarList.list returns 100 entries per page by default and caps at
# 250, handing back a nextPageToken when there are more. Taking only
# the first page would silently hide calendars from the picker, and the
# user would have no way to tell the list was truncated.
entries = []
page_token = None
for _ in range(_CALENDAR_LIST_MAX_PAGES):
response = service.calendarList().list(
maxResults=250, pageToken=page_token).execute()
entries.extend(response.get('items', []))
page_token = response.get('nextPageToken')
if not page_token:
break
else:
# 2500 calendars in, something is wrong with the account or the
# token is looping; show what was collected rather than spin.
logger.warning(
'calendarList paging stopped at %d pages with more remaining',
_CALENDAR_LIST_MAX_PAGES)
calendars = [{
'id': entry.get('id'),
# The picker labels each row with summary and falls back to the id
# only in its own display, so send something either way.
'summary': entry.get('summary') or entry.get('id'),
'primary': bool(entry.get('primary', False)),
} for entry in entries if entry.get('id')]
# Primary first, then alphabetically: the list is usually short but the
# one the user wants is almost always their own calendar.
calendars.sort(key=lambda c: (not c['primary'], c['summary'].lower()))
return jsonify({'status': 'success', 'calendars': calendars})
except Exception as e:
logger.error('Error in list_calendar_calendars', exc_info=True)
return jsonify({'status': 'error',
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)}), 500
@api_v3.route('/plugins/assets/delete', methods=['POST'])
def delete_plugin_asset():
"""Delete an asset file for a plugin"""
@@ -0,0 +1,196 @@
/**
* Google OAuth Widget
*
* Step 2 of the calendar plugin's setup, between uploading the OAuth client
* file and picking calendars. Google will not let a headless device complete
* consent on its own, so the flow is necessarily two calls with a human in
* between:
*
* 1. POST /api/v3/plugins/calendar/authenticate with no body
* -> { auth_url } to open in a browser
* 2. the browser lands on a loopback address that fails to load; its URL
* carries the authorization code. POST it back as redirect_url
* -> the server exchanges it and writes token.pickle
*
* The failed page in step 2 is expected and is worth saying out loud, because
* it looks exactly like something went wrong.
*
* @module GoogleOAuthWidget
*/
(function () {
'use strict';
if (typeof window.LEDMatrixWidgets === 'undefined') {
console.error('[GoogleOAuthWidget] LEDMatrixWidgets registry not found. Load registry.js first.');
return;
}
const ENDPOINT = '/api/v3/plugins/calendar/authenticate';
window.LEDMatrixWidgets.register('google-oauth', {
name: 'Google OAuth Widget',
version: '1.0.0',
/**
* @param {HTMLElement} container
* @param {Object} config - schema config (unused)
* @param {*} value - unused; this widget stores nothing
* @param {Object} options - { fieldId, pluginId, name }
*/
render: function (container, config, value, options) {
const fieldId = options.fieldId;
// Nothing is stored in config by this step -- the result is
// token.pickle on the device -- but the form still expects a field.
const hidden = document.createElement('input');
hidden.type = 'hidden';
hidden.id = fieldId + '_hidden';
hidden.name = options.name;
hidden.value = value || '';
const startBtn = document.createElement('button');
startBtn.type = 'button';
startBtn.className = 'px-3 py-1.5 text-sm rounded-md bg-blue-600 hover:bg-blue-700 text-white';
startBtn.innerHTML = '<i class="fas fa-key"></i> Connect Google Account';
const status = document.createElement('p');
status.className = 'text-xs text-gray-400 mt-2';
// Every message this widget gives -- the consent link is ready,
// the exchange failed -- arrives here after an async call, so a
// screen reader is told nothing unless it is a live region.
status.setAttribute('role', 'status');
status.setAttribute('aria-live', 'polite');
const step2 = document.createElement('div');
step2.className = 'mt-3 hidden';
const link = document.createElement('a');
link.target = '_blank';
link.rel = 'noopener noreferrer';
link.className = 'text-blue-400 underline text-sm break-all';
link.textContent = 'Open the Google consent screen';
// Deliberately loud. After consent the browser is redirected to a
// loopback address nothing is listening on, so it lands on a
// browser error page -- which reads as a failure at exactly the
// moment the user has to act on it. Said quietly in grey it gets
// missed, and the flow looks broken when it is working.
const hint = document.createElement('div');
hint.className =
'mt-3 p-3 rounded-md border border-amber-500/60 bg-amber-500/10';
hint.innerHTML =
'<p class="text-sm text-amber-300 font-semibold">'
+ '<i class="fas fa-triangle-exclamation"></i> '
+ 'The next page will fail to load. That is expected.</p>'
+ '<p class="text-xs text-amber-200/90 mt-1">'
+ 'After you approve access, Google sends your browser to '
+ '<code>127.0.0.1</code>, where nothing is running \u2014 so you will see '
+ '"This site can\u2019t be reached" or similar. Nothing has gone wrong. '
+ 'Copy the <strong>entire address</strong> out of the address bar '
+ '(it contains <code>?code=...</code>) and paste it in the box below.</p>';
const codeInputId = fieldId + '_redirect_url';
const codeLabel = document.createElement('label');
codeLabel.className = 'block text-xs text-gray-300 mt-3';
codeLabel.textContent = 'Paste the address from that failed page here:';
// The label was visible but not associated, so the input still had
// no accessible name -- a placeholder is not one, and it vanishes
// on focus, which is exactly when the value is being pasted.
codeLabel.setAttribute('for', codeInputId);
const codeInput = document.createElement('input');
codeInput.type = 'text';
codeInput.id = codeInputId;
codeInput.placeholder = 'http://127.0.0.1/?code=...';
codeInput.className =
'mt-2 block w-full px-3 py-2 text-sm border border-gray-600 '
+ 'rounded-md bg-gray-800 text-gray-100';
const finishBtn = document.createElement('button');
finishBtn.type = 'button';
finishBtn.className = 'mt-2 px-3 py-1.5 text-sm rounded-md bg-green-600 hover:bg-green-700 text-white';
finishBtn.innerHTML = '<i class="fas fa-check"></i> Finish Authentication';
function say(message, kind) {
status.textContent = message;
status.className = 'text-xs mt-2 ' + (
kind === 'error' ? 'text-red-400'
: kind === 'success' ? 'text-green-400'
: 'text-gray-400');
}
function post(body) {
return fetch(ENDPOINT, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body || {})
}).then(function (r) {
return r.json().catch(function () {
// A non-JSON body here means the request never reached
// the handler -- worth saying so rather than "undefined".
return { status: 'error', message: 'Server returned ' + r.status };
});
});
}
startBtn.addEventListener('click', function () {
startBtn.disabled = true;
say('Requesting a consent link...');
post({}).then(function (data) {
startBtn.disabled = false;
if (data.status !== 'success' || !data.auth_url) {
say(data.message || 'Could not start authentication.', 'error');
return;
}
link.href = data.auth_url;
step2.classList.remove('hidden');
say(data.message || 'Open the link, approve, then paste the address back.');
}).catch(function (err) {
startBtn.disabled = false;
say('Request failed: ' + err.message, 'error');
});
});
finishBtn.addEventListener('click', function () {
const pasted = codeInput.value.trim();
if (!pasted) {
say('Paste the address your browser was redirected to.', 'error');
return;
}
finishBtn.disabled = true;
say('Exchanging the code with Google...');
post({ redirect_url: pasted }).then(function (data) {
finishBtn.disabled = false;
if (data.status !== 'success') {
say(data.message || 'Authentication failed.', 'error');
return;
}
say(data.message || 'Authenticated.', 'success');
step2.classList.add('hidden');
codeInput.value = '';
}).catch(function (err) {
finishBtn.disabled = false;
say('Request failed: ' + err.message, 'error');
});
});
step2.appendChild(link);
step2.appendChild(hint);
step2.appendChild(codeLabel);
step2.appendChild(codeInput);
step2.appendChild(finishBtn);
container.appendChild(hidden);
container.appendChild(startBtn);
container.appendChild(status);
container.appendChild(step2);
},
getValue: function (fieldId) {
const hidden = document.getElementById(fieldId + '_hidden');
return hidden ? hidden.value : '';
}
});
})();
+1
View File
@@ -987,6 +987,7 @@
<script src="{{ url_for('static', filename='v3/js/widgets/custom-feeds.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/array-table.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/google-calendar-picker.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/google-oauth.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/day-selector.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/time-range.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/time-picker.js') }}" defer></script>
@@ -815,7 +815,7 @@
<i class="fas fa-info-circle mr-1"></i>
Changes in the file manager save immediately — no need to click Save Configuration.
</p>
{% elif str_widget in ['text-input', 'textarea', 'select-dropdown', 'toggle-switch', 'radio-group', 'date-picker', 'time-picker', 'slider', 'color-picker', 'email-input', 'url-input', 'password-input', 'font-selector', 'file-upload-single', 'plugin-file-manager'] %}
{% elif str_widget in ['text-input', 'textarea', 'select-dropdown', 'toggle-switch', 'radio-group', 'date-picker', 'time-picker', 'slider', 'color-picker', 'email-input', 'url-input', 'password-input', 'font-selector', 'file-upload-single', 'plugin-file-manager', 'google-oauth'] %}
{# Render widget container #}
<div id="{{ field_id }}_container" class="{{ str_widget }}-container"></div>
<script>