Compare commits

...
Author SHA1 Message Date
ChuckBuildsandClaude Opus 5 5a6dfbfc9a fix(web): sample both sides of DST when comparing zones
CodeRabbit caught this and it is right: comparing the wall clock at one
instant treats zones that merely coincide right now as the same one.
America/New_York and America/Lima hold the same offset all winter, so a
panel set to the wrong one of the two ticked the step in January and then
ran an hour off from March -- a silent false pass, which is the failure the
whole check exists to prevent. Same shape as the dateStyle problem in the
previous commit: a comparison coarser than it looks.

Three instants now, all of which must agree: now, and mid-January and
mid-July of the current year. Those sit either side of DST in both
hemispheres, so only zones that agree year-round match. Toronto still
matches New York, which is correct -- either renders the same times.

Two tests. A static one asserts the comparison samples more than the
current instant, since reverting to `[now]` looks like a simplification.
And a table pinning which pairs must count as the same zone: aliases and
same-rule zones equal, seasonal coincidences (New York/Lima,
Phoenix/Los_Angeles, Sydney/Guadalcanal) not. That table mirrors the
algorithm rather than executing the shipped JS -- there is no JS runtime
here and the repo has no JS test infra -- so it records the verdicts the
browser code has to reach, and the static guard keeps the two aligned.

Mutation-checked: reverting to a single instant fails the static guard.

Also documented what the city test compares. CodeRabbit read it as always
failing, on the grounds that the label differs between Tampa and Seattle.
It does, but timezone_step() returns the opening tag only, so the
comparison is over data-done and data-tz and the label is not in it. The
assertion is left as an equality over the whole tag, which is stronger than
checking the two attributes by name; the docstring now says so.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-17 15:53:43 -04:00
ChuckBuildsandClaude Opus 5 dcba6120f9 fix(web): compare zones on fields Intl has always had
dateStyle/timeStyle are late additions to Intl -- Firefox shipped them in
91 -- and an implementation that does not know them ignores them and
formats the date alone. The comparison would then read New York, Chicago
and Madrid as the same zone and tick the step for a timezone that is
plainly wrong, which is the failure the check exists to catch. Silent, and
only on older browsers.

Explicit numeric fields (year/month/day/hour/minute) have been in Intl
since ECMA-402 v1, so there is nothing left to degrade to.

The options look like a stylistic choice, so a test pins them: it reads the
comparison with comments stripped -- the comment names dateStyle to explain
why it is not used -- and fails if either style option comes back or a
time field is dropped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-17 15:43:38 -04:00
ChuckBuildsandClaude Opus 5 2f64dbc48c fix(web): verify the onboarding timezone step, don't compare it to the default
The Getting Started card's timezone step ticked when the saved timezone
differed from the value config.template.json ships (America/New_York),
OR-ed with the saved city differing from Tampa. Both halves were wrong.

"Differs from the default" answers "did somebody edit this?", but what the
checklist needs to know is whether the value is right. A user genuinely in
America/New_York could never satisfy it, so the card nagged forever with
four of five steps done -- the case that prompted this, on a panel whose
timezone was correct all along.

The city half was worse than useless: the saved city says nothing about
whether the timezone is set, and because the two were OR-ed, saving a city
ticked the step off with the timezone still wrong. That is the direction
that actually breaks displays, since event times then render in the wrong
zone.

The browser already knows its own zone, so compare against that. No new
persisted state, no network, and it catches the reverse case the old test
got backwards: a panel still set to the old zone after a move now stays
unticked, where before it ticked the moment the value stopped being the
default. Zones are compared by the wall-clock time they produce for one
instant rather than by identifier, so aliases (Asia/Calcutta vs
Asia/Kolkata, Europe/Kiev vs Europe/Kyiv) don't read as a mismatch. When
they genuinely differ the step names the browser's zone, so an unticked box
says why. Configs with no timezone, an unparseable zone, or a browser
without Intl leave the step open for the existing manual tick.

The step still deep-links to the General tab, and the location value stays
visible in its label -- it just no longer votes on whether the timezone is
configured.

Tests render the partial across configured zones and both cities: the step
never pre-ticks server-side, carries the configured zone for the client to
check, is unmoved by the city, and the panel-size step still resolves
server-side. Reverting the template fails 9 of the 11.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-17 11:36:08 -04:00
08265c1135 feat(vegas): let live content keep its place in the ticker (#457)
* feat(vegas): let live content keep its place in the ticker

Live content used to preempt Vegas outright: while any plugin reported
live priority the display controller refused to run the ticker at all and
showed a full-screen scoreboard instead. Keeping the marquee meant not
seeing live scores; seeing live scores meant losing the marquee.

Two changes, both off by default.

vegas_scroll.live_in_ticker keeps the ticker running through a live game.
Three places assumed the takeover and all three now honour it: the
controller's gate, the coordinator's per-frame pause, and the rotation
switch that would otherwise move current_mode_index underneath a ticker
that never yields.

And the rotation is no longer a strict round robin. It was one slot per
plugin per cycle, so with a dozen plugins enabled a live score came round
once a lap and could be minutes old on screen. A plugin can now hold
several slots, placed by Smooth Weighted Round-Robin -- the same
scheduler the sports plugins already use to rotate their own games. The
property that matters is that repeats are spread through the cycle
rather than clumped: three in a row and then silence would be worse than
no boost at all.

Weight comes from the plugin first, via a new optional
get_vegas_priority_weight(), then from the core: live content earns
live_weight, everything else 1. So existing plugins gain the behaviour
without changes, and the hook exists for the one thing the core cannot
work out -- the core can see that a game is live but not whose, so only
the plugin can say a favorite is playing.

Documented in ADVANCED_FEATURES (worked example, why weights are per
plugin not per game, and that frequency is not freshness),
CONFIG_REFERENCE, PLUGIN_API_REFERENCE, and the config template.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(vegas): carry the new keys through config, and correct two docs

Three findings from CodeRabbit, all valid.

to_dict() and update() enumerate keys explicitly and had not learned the
three new ones, so get_status() never reported them and a live config
change never applied -- turning live_in_ticker on in the web UI would
have done nothing until a restart. update() clamps the weights exactly
as from_config does.

The vegas_scroll key count in ADVANCED_FEATURES said 29; the template
has 30. My arithmetic, not the reviewer's.

The third was a documentation error rather than a code one, and I have
fixed it the other way round. The docs claimed a raising
get_vegas_priority_weight() is treated as weight 1. The code instead
falls through to the core's own live-content check, and that is the
better behaviour: the hook is only how a plugin asks for *more* than
live_weight, and has_live_priority/has_live_content are separate methods
guarded separately, so a plugin with a broken weight calculation should
lose the favorite distinction and keep the live boost. Said so in the
code, the base-plugin docstring and the API reference.

The test fake now fails in each place independently, because the two
failures mean different things: a broken hook still earns live_weight, a
plugin that cannot say whether it is live has nothing to fall back on
and weighs 1.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ui/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(vegas): stop the heaviest plugin doubling across the cycle seam

Smooth Weighted Round-Robin spaces repeats well within a pass, but it
schedules the heaviest item first and usually last as well. The strip
loops, so those two are neighbours: the marquee showed the same plugin
twice running at exactly the one join a within-cycle check cannot see.
Observed on a live rig at 28 slots -- gaps of 6, 7, 7, 7 and then 1.

Rotating the list does not fix it. Rotation preserves the cyclic order
exactly, so it moves where the seam is drawn rather than the adjacency
itself; the trailing entry has to be swapped with one from the middle.

The first version swapped with the first slot that merely fitted, which
undid the spacing this exists to protect -- it moved a repeat from a gap
of 7 into a gap of 2, more clumped than the seam had ever been. It now
picks the candidate furthest from any other appearance, so the repeat
lands in the widest gap.

Left alone when no candidate exists. A plugin holding most of the slots
has to neighbour itself, and scheduling it is better than refusing to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(vegas): stop the seam repair creating the duplicate it removes

Swapping the trailing repeat with a middle slot moves two elements, and
the candidate filter only guarded one of them. It checked the neighbours
`repeated` would acquire at j, but not what the displaced element would
sit beside at the end -- so ['a','b','c','d','x','y','x','a'] came back
as [...,'x','x'], the seam duplicate traded for a fresh one. Reported by
CodeRabbit with that exact case.

Adding the missing condition fixed it and immediately broke something
else: schedule[j] is schedule[-2] when j is the second-to-last slot, so
that candidate was always excluded, and ['a','b','c','a'] lost the only
repair it has. The same class of mistake twice, from reasoning about
which neighbours two moved elements end up with.

So it no longer reasons. It performs each candidate swap, counts the
cyclic duplicates in the result, and keeps the best one that has none --
preferring whichever leaves the boosted plugin most evenly spread. When
no such swap exists the schedule is returned untouched, which is the
unavoidable case: a plugin holding most of the slots has to neighbour
itself.

Fuzzed across 6,956 seam schedules: none made worse, none lost an entry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 17:02:36 -04:00
5713fd20a7 fix(calendar): implement the OAuth and calendar-listing endpoints (#458)
* fix(calendar): implement the OAuth and calendar-listing endpoints

The plugin's config advertised a three-step setup and only step 1
existed. Step 3's picker fetched
/api/v3/plugins/calendar/list-calendars, which was never registered, so
Flask fell through to the global 404 handler and the user saw "Resource
not found" -- a message that names nothing and points nowhere. Step 2
had no endpoint at all, so even a working picker would have found no
token to list with.

Two routes, following the pattern the spotify and ytm plugins already
use for their own auth scripts:

  POST /plugins/calendar/authenticate    two-step Google OAuth
  GET  /plugins/calendar/list-calendars  calendars for the picker

The authenticate route drives calendar_registration.py, which the plugin
already ships and which was written expressly for this -- it reads a
redirect URL on stdin and prints one JSON object. It takes two calls
because a human has to visit Google in between; the script persists the
PKCE verifier from the first call for the second, without which the
exchange fails with "Missing code verifier".

The listing route reads the token directly rather than shelling out
again: the picker is interactive and a subprocess per click is slower
than the API call it would wrap. It refreshes an expired token in place,
sorts the primary calendar first, and drops entries with no id, which
could not be selected anyway.

Both name the plugin when it is not installed, rather than reproducing
the anonymous 404 that started this.

Verified against the live Google API on the dev rig: HTTP 200 with the
account's real calendars.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(calendar): one input for the auth code, and a louder warning

Two things reported after testing the flow.

There were two boxes and no way to tell which to use. The config
template's string branch dispatches widgets from an allow-list of names,
and anything missing from it falls through to a plain input type=text --
so the field rendered both the widget's own box and a stray one for the
same key. google-oauth is now on that list, which is all the widget ever
needed to render in place of the fallback rather than beside it.

And the warning that the redirect page fails to load was small grey text
under a link, which is where it is least likely to be read. It is now an
amber callout that leads with "The next page will fail to load. That is
expected." The failure lands at exactly the moment the user has to act
on it, and it looks precisely like the flow breaking rather than
working. The paste box is labelled too, rather than relying on a
placeholder that vanishes on focus.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(calendar): redact script diagnostics, and page the calendar list

Three findings from CodeRabbit, all valid.

Raw subprocess output was being returned to the client -- the script's
stderr on one path, and its own error payload on another. CodeQL flagged
the same line. That script handles OAuth client secrets and interpolates
exceptions into its messages, so either could carry a secret or a path.
Both now go to the log unredacted, where they are worth having in full,
and reach the client through a redactor.

That redactor already existed inside describe_exception, which only
takes exceptions. Split out as redact_text: an exception is not the only
thing worth returning, and a subprocess's stderr is just as capable of
quoting a token.

calendarList.list returns 100 entries per page by default, caps at 250,
and hands back a nextPageToken when there are more. Reading one page
would have hidden calendars from the picker with nothing to say the list
was cut short. It now pages, asking for 250 at a time, bounded at ten
pages so a malformed token cannot spin.

And a test helper was a lambda where ruff wants a def.

The five new tests fail against the previous commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(calendar): redact the last two raw exception interpolations

CodeQL flagged four exposure paths. Two were mine and genuinely raw: the
OSError from failing to spawn calendar_registration.py, which carries the
interpreter path and whatever the OS chose to say, and the ImportError
for the Google libraries, whose message named the missing module by
interpolating the exception directly. Both now go through
describe_exception, and the unredacted text goes to the log.

The other two are the repo-wide pattern from PR #448 -- 67 handlers on
main already return details=describe_exception(e), and these two new
handlers follow it. That function is the sanitizer: it strips URL
userinfo, auth headers and credential-shaped key=value pairs, collapses
to one line and caps the length. CodeQL's taint tracking cannot see a
sanitizer it has no model for, so it reports the flow regardless.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(calendar): announce status changes, name the paste box, drop a no-op

Three more from the review, all valid.

The status line is written after every async call -- the consent link is
ready, the exchange failed -- and was a plain paragraph, so a screen
reader was told none of it. It is a live region now.

The paste box had a visible label that was never associated with it, so
its only accessible name was the placeholder, which disappears on focus:
precisely when the value is being pasted. The label now points at the
input by id.

And a conditional in the test helper returned the same value from both
branches, which Ruff flags as RUF034. It was left over from making the
fake page; one page is all those cases need, and TestPagination builds
its own sequences.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* test(calendar): assert the accessibility relationships, not their parts

The previous assertions searched for role="status", aria-live, a label
`for` and an input `id` independently, so they passed whether or not
those belonged together. Two attributes on different elements announce
nothing, and a `for` that names something other than the input leaves it
just as anonymous.

Both attributes are now asserted on the status element itself, and the
label and input are checked to go through the same identifier rather
than merely both existing. Verified by mutation: a mismatched pair and a
displaced aria-live are both caught.

Reported by CodeRabbit, against tests I had written two commits earlier.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 15:25:43 -04:00
9cf30bbbef fix(startup): bound the initial plugin update so the panel lights sooner (#456)
* fix(startup): bound the initial plugin update so the panel lights sooner

DisplayController.__init__ calls _update_modules() once to populate
plugin data before the first frame. It walks every loaded plugin in
turn, and each update blocks the calling thread for up to the executor's
30s timeout, so the uncapped total is the sum of every slow plugin on
the system. The rig's own log:

    Initial plugin update completed in 82.255 seconds
    Initial plugin update completed in 55.123 seconds
    Initial plugin update completed in 25.975 seconds

The panel shows nothing for all of it.

Nothing is lost by stopping early. A plugin that has never updated is
immediately due, so run_scheduled_updates() collects it seconds later --
with the display already running rather than blank.

A deadline alone was not enough: it is checked before each plugin, so
the last one to start could still block for the full 30s, and a 20s
budget produced a 31.8s pass on the rig. The remaining budget is now
passed down as that update's timeout too, with a floor so a plugin
starting on the last sliver is not handed ~0s and recorded as having
timed out for a slot it never had. Measured after: 20.006s.

Found while profiling a scroll freeze with py-spy, which caught the main
thread 9.34s inside execute_with_timeout's join. Worth being clear that
this is startup latency, not the recurring stutter -- _update_modules
has exactly one caller and runtime updates already run off the display
thread.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* feat(display): show the device address on the startup screen

That screen is what the panel holds for the whole initial plugin update,
and on a headless Pi it is the only place the address appears without
going looking for it -- so it now carries the address under
"Initializing".

The lookup connects a UDP socket, which sends no packets: it only asks
the kernel which source address it would route from. That costs 0.03ms
and works with the network down so long as a route exists. Deliberately
not `hostname -I` plus a systemctl probe for AP mode, which is how the
web launcher does it -- two subprocesses with multi-second timeouts, on
the startup path this branch exists to shorten.

Two things had to change for the address to be worth putting there.

The text is now sized to fit rather than fixed at 8px: "Initializing" is
96px in PressStart2P, drawn at x=10, so it already ran off the side of a
64px panel before an address was added. It falls back to 4x6 where that
does not fit, and both lines are centred.

And the test pattern is punched out from behind the block, with the text
drawn white rather than blue. The diagonal runs through the middle of
the panel, which is exactly where this sits, and blue on black reads
fine on a monitor but is marginal on a dim panel. An address that cannot
be read off the wall is not worth showing.

The rendering tests assert against pixels -- no green left behind the
text at any supported size, enough lit pixels to be visible -- rather
than against the geometry that produced them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(display): keep the startup text blue -- it is a channel reference

The test pattern lights one pure channel per element: red border, green
diagonal, blue text. That is how a glance at the panel tells you whether
led_rgb_sequence is right -- wire it BGR and the border comes up blue
and the text red. Drawing the text white, as the previous commit did for
contrast, lights all three channels and destroys the only blue reference
on the screen.

Reverted to blue, with the reason written down so it is not treated as a
style preference again, and with tests that pin it: the text must be
pure blue, nothing on the screen may be white, and all three primaries
must be present.

The punched-out backdrop stays. It only removes the diagonal from behind
the glyphs, which costs nothing diagnostically -- the diagonal is still
plainly visible across the rest of the panel -- and it is what makes the
address readable at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(startup): defer a plugin with too little budget, rather than clamp it

The per-plugin timeout was clamped up to a floor, so a plugin that began
with a sliver of budget left was granted the full floor and ran on past
the deadline: a 20s budget could take 22. The floor existed to stop a
plugin being handed a slot too short to use and then recorded as having
timed out, which is a real concern, but clamping solved it by breaking
the bound.

Deferring solves both. Below the floor the plugin is left to the update
tick, which was already the fate of everything after the deadline, so
nothing new is lost -- a plugin that has never updated is immediately
due. Above it, the timeout is the exact remainder, and the pass cannot
outlast its deadline.

Measured on the rig after the change: 20.002s, 5 plugins deferred.

Also names an unused binding in the initializing-screen test.

Both reported by CodeRabbit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 14:58:05 -04:00
a51fb7ce11 feat(vegas): make scroll stutter visible, and catch it in the act (#454)
* feat(vegas): make scroll stutter visible, and catch it in the act

The loop reported only a mean FPS over a five-second window. At 120fps
that is ~600 frames, so a 200ms freeze -- plainly visible on a marquee --
moves the average from 120.0 to 115.4 and reads as healthy. Stutter was
literally unmeasurable.

The FPS line now carries p99, the worst frame, and a hitch count. On the
dev rig that immediately turned "it sometimes stutters" into a number:
two freezes of 3.2s and 0.7s in twenty minutes, with every other frame
under 81ms.

Statistics say a stall happened but not what caused it, and by the time
they are logged the stack is gone. So there is also a watchdog that dumps
every thread's stack while the loop is still wedged. It is off unless
LEDMATRIX_STALL_WATCHDOG is set to a threshold in seconds, since it
prints a lot. Pointed at the 3.2s freeze it named the culprit on the
first try: a plugin generating a 17,000px scroll image, logo PNG decode
and all, synchronously on the render thread.

The hitch threshold is relative to what frames actually cost, not to the
configured target. The target is routinely set above what the panel can
hold so vsync does the pacing; measured against that budget every
ordinary frame counts as a hitch, and the first version of this counter
duly reported 250 per window on a display running perfectly smoothly.

The watchdog is owned by the coordinator, not created per iteration --
run_iteration is called repeatedly, so building one there would leak a
thread each time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(vegas): let the watchdog see stalls that hold the GIL

The watchdog only noticed a late heartbeat, which a whole class of
freeze can never produce: if the loop is inside one long C call that
holds the GIL, this thread cannot run during the stall, and by the time
it does the loop has already checked in. On the dev rig that hid a
recurring 3.2s freeze completely -- twenty minutes of watching produced
one dump, for an unrelated 0.4s stall.

What it can still observe is that its own sleep ran long. A badly
overshot wait is now reported as a stall in its own right. The stacks
are stale by then and the message says so, but knowing the freeze is
GIL-holding is most of the diagnosis: it rules out lock contention and
scheduling, and points at a single long C call.

This also explains why lowering sys.setswitchinterval changed nothing --
the switch interval cannot preempt a C call that never releases the GIL.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* feat(vegas): report the worst frame, not just the mean

The loop logged only a mean FPS over a five-second window. At 120fps
that is ~600 frames, so a 200ms freeze -- plainly visible on a marquee
-- moves the average from 120.0 to 115.4 and reads as perfectly healthy.
Stutter was unmeasurable, which is why "it sometimes freezes" went
unpinned for so long.

Adding p99 and the worst frame turned that into a number immediately: on
the dev rig, two freezes of 3.2s and 0.7s in twenty minutes with every
other frame under 81ms. Not general slowness -- two rare, total stalls,
which is a different problem with a different fix.

Costs 0.96us per frame, about 0.012% of an 8.3ms frame.

This replaces an earlier version that also shipped a stall watchdog and
a hitch counter. The watchdog never found anything -- one dump in
forty-five minutes, for an unrelated stall -- because it can only notice
a late heartbeat, and the freeze happens in coordinator.start() before
the frame loop begins beating. py-spy found the cause in one recording
by sampling the process externally, which needs no code here. The hitch
counter went with it: it needed a rolling median every frame, which was
most of the cost, to produce a number the worst frame already tells you.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(vegas): use the nearest-rank index for p99

int(n * 0.99) is off by one, and at exactly 100 samples it selects the
maximum -- which is the number logged immediately beside it as the worst
frame. The two columns exist to say different things, p99 the
bad-but-ordinary frame and worst the outlier, so they agreed precisely
when the sample was smallest and least informative.

Nearest rank is ceil(n * fraction) - 1. Extracted so it can be tested
directly rather than only through a five-second logging interval.

Reported by CodeRabbit on the PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 14:06:32 -04:00
fce1fdac57 Add panel orientation setting for upside-down mounting (#455)
Adds a display.hardware.orientation config field ("normal" / "180")
so panels mounted upside down (e.g. to put the Pi/wiring on a more
convenient side) render correctly without custom pixel_mapper_config
edits. Composes onto the existing pixel_mapper_config as a trailing
"Rotate:180" mapper, so it stays independent of any custom mapper
string (e.g. U-mapper chain layouts) already in use.

Exposed as a "Panel Orientation" dropdown in the web UI's Display
settings, validated server-side, and documented in README and
CONFIG_REFERENCE.


Claude-Session: https://claude.ai/code/session_01FakipqMDHQLpsFjTuBdSFQ

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-12 09:40:48 -04:00
7171e6c022 fix(cache): one cleanup thread per cache directory, not per manager (#453)
The display process ran three cleanup threads over one directory:

    14:22:59.954  display_controller        (the real manager)
    14:22:59.973  startup validation, run 1 (discarded)
    14:23:01.055  startup validation, run 2 (discarded)

Two of those managers existed only to read a directory path.
StartupValidator._validate_cache_directory built a whole CacheManager to
call get_cache_dir(), and validation runs twice -- once before the
plugin manager exists and again after. Each construction also probes
writability by writing and deleting .writetest on the card.

The discarded ones never went away. cleanup_loop closes over `self`, so
the thread keeps its manager alive: two objects that could never be
collected, waking every 24 hours to re-scan the same 9,000-file
directory. Nothing stopped them either -- stop_cleanup_thread had no
callers anywhere in the tree.

Two changes. The validator now takes the CacheManager the application
actually uses, which is also the more correct thing to validate; when
no caller supplies one it still builds its own, but stops the thread
afterwards. And CacheManager now tracks which directory it is sweeping,
so the second manager over a directory skips starting a thread at all.
That is the right granularity regardless of call sites: the sweep lists
a directory and deletes from it, so a second thread only duplicates the
scan. Ownership is released on stop, so a survivor can take over rather
than leaving the directory permanently unclaimed by a dead owner.

Measured directly, three managers over one directory: 3 threads before,
1 after.


Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 08:42:56 -04:00
9fbdd71941 fix(cache): collect the temp files abandoned writes leave behind (#452)
* fix(cache): collect the temp files abandoned writes leave behind

DiskCache.set() writes through mkstemp then os.replace, and removes its
own temp file in a finally. That covers a write that fails, but not a
process that dies between the two -- a SIGKILL, a lost restart race, a
power cut, all ordinary on a Pi.

Nothing ever collected what was left. The temp names are
".<key>.json.<random>", and cleanup_expired_files listed only names
ending in .json, so every one of them was invisible to the sweep for as
long as the card had been in service. On the dev rig: 76 files,
1,050 MB, 81% of the whole cache directory, the oldest six months old.
The startup sweep reported "18/8864 files deleted, 0.01 MB freed" while
sitting on top of a gigabyte it could not see.

They are removed after an hour. A real write holds its temp file for
milliseconds, so that is far outside any in-flight write while still
clearing the same day's debris, and it is deliberately not tied to the
retention policies: those say how long data stays useful, and a
half-written file never was.

The predicate is tested harder than the sweep, because a false positive
deletes real data. It matches the shape set() creates rather than just a
leading dot, so a completed ".json", a stray .gitignore, and a
"weather.json.bak" are all left alone -- and one test drives set()
itself and asserts the names it produces are matched, so the writer and
the predicate cannot drift apart.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(cache): count swept temp files as scanned

files_scanned only counted completed .json files, so a sweep that
removed orphans reported more deleted than it had looked at -- the
summary line renders "<deleted>/<scanned>", which came out as "76/1".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 08:40:07 -04:00
2add759f40 fix(odds): identify the odds requests to ESPN (#451)
The odds fetch used a bare requests.get, so it went out as
python-requests/x.y -- the one agent ESPN is known to reject. Around
2026-08-04 it began 403ing browser strings and bare custom tokens alike;
what it accepts is a token carrying a URL that says who is calling.
Every other ESPN caller in the tree already sends that header
(src/common/api_helper.py, src/base_classes/data_sources.py); this path
was simply missed.

It is the worst one to miss. Odds are fetched per live game from inside
the live update loop, so its failures are the ones that cost the caller
its whole update budget -- the same path the 5s timeout and the cooldown
were added to protect.

Sent via a session rather than per-call, which also reuses the
connection across a slate. Deliberately no retry adapter, unlike
api_helper: retries multiply request_timeout, which is 5s precisely to
stay inside the 30s operation budget.

The existing tests patched the module's requests.get, which this change
bypasses -- test_base_odds_manager was consequently reaching the real
ESPN and taking 404s. Both files now patch the session, and the new
tests pin the agent against api_helper's live value so the two cannot
drift apart the next time ESPN moves the goalposts.


Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 16:10:56 -04:00
bb1a1671ec fix(cache): make the ttl parameter actually control expiry (#450)
CacheManager.set(key, data, ttl=...) stored the number and no read path
ever consulted it. Expiry came from a max_age inferred from substrings
in the key -- "live", "odds", "stock" -- so all 52 callers passing a ttl
were writing a value that did nothing. The docstring said so outright:
"stored for compatibility but expiration is still controlled via max_age
when reading". It is easier to read that as a note than as a defect,
which is presumably how it survived.

Both cache layers already hold the record when they decide, so each now
prefers an explicit ttl and falls back to max_age when there is none.
The caller that wrote the record knows what its data is; a substring
guess is a reasonable default for records that never said, and a poor
override for records that did.

Measured against a device's real cache of 8,875 entries carrying a ttl,
the inferred and intended values disagreed nearly everywhere:

    stocks    max_age  600  vs ttl    1800   4903 entries
    news      max_age 3600  vs ttl     600   1770 entries
    odds      max_age 1800  vs ttl    3600   1301 entries
    images    max_age  300  vs ttl 2592000     20 entries

In every case the ttl matches what the plugin plainly intended: stock
quotes cached for half an hour rather than ten minutes, headlines
refreshed every ten minutes rather than hourly, bird photographs that
never change kept for a month rather than five minutes.

Two things make this safe to land now. No sports_live entry carries a
ttl at all -- the live-score path does not use set(ttl=) -- so live
freshness is untouched, which matters with a season two weeks out. And
replaying the change against that real cache, 997 currently-expired
entries become live while not one live entry becomes expired, so there
is no invalidation spike on deploy.


Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 14:14:25 -04:00
8159afca43 fix(odds): stop a stalled ESPN taking the whole plugin update with it (#449)
Odds are fetched per live game from inside SportsLive.update(), with
show_odds defaulting on, and the plugin executor kills an operation at
30s. The odds request timeout was also 30s, so a single stalled request
consumed the entire budget and the update carrying every game's score
was killed.

Out of season that is invisible: preseason week 1 returns one game. A
Sunday slate is around sixteen, so the odds of at least one slow request
rise sharply just as the cost of losing the update does.

Shorten the request timeout to 5s, and after a network failure skip the
network for 60s. The timeout alone is not enough -- sixteen consecutive
5s timeouts still blow through -- and when ESPN is unreachable it is
unreachable for the whole slate, so the first failure already answers
the question for the rest of the pass.

    before: one stalled request = 30s = the entire budget
    after : 5s, the rest of the slate skipped, retry after 60s

The stale-cache fallback is unchanged: the cache is consulted before any
of this, and the failing request still falls back to it.

An earlier version of this branch also jittered the cache TTL to stagger
expiry across a slate. That has been dropped: CacheManager.set() stores
ttl for compatibility but the read path expires entries by a per-type
max_age (1800s for odds), so the jitter was inert. Making the read path
honour a per-entry ttl is a real fix but changes a contract 48 plugin
call sites already rely on, which is not a change to make two weeks
before the season.


Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 13:56:53 -04:00
44f59ede07 fix(web): say what actually went wrong instead of "unknown" (#448)
* fix(web): say what actually went wrong instead of "unknown"

Every failing endpoint returned "An error occurred; see logs for
details" and nothing else. That is survivable until the logs are the
thing you cannot reach: a device whose SD card was failing answered the
restart action, /system/status and /logs with that same sentence -- the
log viewer included, because journalctl could not be executed -- while
the exception underneath said

    [Errno 5] Input/output error: 'systemctl'

which names the fault outright. The only endpoint that helped was
/health, and only because it happens to pass a subprocess's stderr
through. Diagnosis came down to guessing which endpoint leaked something.

Add describe_exception(), returning "TypeName: message" on one line, and
populate the `details` field that the response schema has always had and
nothing ever filled. The type alone carries information -- a bare
PermissionError says more than any generic sentence.

Exception text is not automatically safe to echo: a requests error
quotes the URL it failed on, and plugins that authenticate by query
string put their key there. Credential values are redacted while the
parameter name is kept, since knowing which credential was involved is
part of the diagnosis. Length is capped and newlines collapsed so a
parser's context cannot flood a JSON field.

Nine handlers in api_v3 bound the exception and never used it, so the
promised log entry was never written either -- "see logs for details"
was false, not merely unhelpful. Those now log with a traceback and
carry the detail. The other 60 already logged and are unchanged; they
can adopt the helper as they are touched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(web): redact auth headers and URL userinfo, and cover every handler

Three review findings.

The sanitizer missed two credential shapes that requests puts in its
exception text verbatim: `Authorization: Bearer <token>` and
`https://user:password@host`. Both would have gone straight into a
response. The auth-scheme name and the username are kept -- they say
which credential and whose without being the secret.

The AST test only asked whether *something* had been logged, so a
`logger.info("failed")` satisfied it while discarding the exception just
as completely. It now requires an error-level record carrying exc_info
and `describe_exception()` called on the handler's own bound exception.

Enforcing that revealed the first cut had scoped itself wrongly. I had
converted the nine handlers that logged nothing and left the sixty that
logged, reasoning their detail was at least in the journal. But
/system/status is one of the sixty, and on the failing device it told me
nothing -- the journal was exactly what could not be read. Splitting
them left most of the diagnostic surface unhelpful for the case this
change exists for, so all sixty-nine now carry the detail.

Two handlers had no bound exception name, and three passed the message
through a variable rather than a literal; both shapes needed doing by
hand. Full suite: 2383 passed, one pre-existing unrelated failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(web): stop reporting client errors as server faults

Werkzeug's HTTPExceptions subclass Exception, so the catch-all handler
saw them too and turned every 405, 400, 413 and 415 into a 500
UNKNOWN_ERROR. A GET on a POST-only route answered "an error occurred;
see logs for details", which tells the caller nothing and blames the
wrong side -- found while probing a device whose POST-only config
endpoints did exactly that.

Hand HTTPExceptions back as themselves, with their own status and
description. A genuine server fault still reports as one, with the
detail this branch adds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(web): redact any auth scheme, and require the detail in the response

Two review findings.

The auth-header pattern listed Bearer, Basic, Digest and Token, so
`Authorization: ApiKey SECRET` or `Negotiate SECRET` went to the client
intact. A fixed list silently leaks whatever it does not name, and
plugin APIs invent their own schemes, so match any scheme name and keep
it while redacting the credential.

The AST test accepted a describe_exception(e) call anywhere in the
handler, which a handler could satisfy by computing the detail and
dropping it before returning the generic message. It now requires the
call inside every return expression, which is where it has to be to
reach the caller.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 13:56:16 -04:00
37 changed files with 3915 additions and 116 deletions
+8
View File
@@ -600,6 +600,14 @@ These settings are typically only needed for non-standard panels or custom confi
- Leave empty unless you need custom mapping - Leave empty unless you need custom mapping
- See rpi-rgb-led-matrix documentation for full options - See rpi-rgb-led-matrix documentation for full options
- **`orientation`** (string, default: "normal")
- Rotates the rendered image to match how the panel is physically mounted
- Set to `"180"` (or use the "Upside Down" option in the web UI's Display
settings) if the panel is mounted upside down — useful for optimizing
where the Raspberry Pi and wiring sit relative to the mounting location
- Applied independently of `pixel_mapper_config` (appended as a trailing
`Rotate:180` mapper), so custom mapper configs keep working alongside it
- **`row_address_type`** (integer, default: 0) - **`row_address_type`** (integer, default: 0)
- How rows are addressed on the panel - How rows are addressed on the panel
- Most panels use 0 (direct addressing) - Most panels use 0 (direct addressing)
+4
View File
@@ -112,6 +112,7 @@
"led_rgb_sequence": "RGB", "led_rgb_sequence": "RGB",
"limit_refresh_rate_hz": 100, "limit_refresh_rate_hz": 100,
"pixel_mapper_config": "", "pixel_mapper_config": "",
"orientation": "normal",
"row_address_type": 0, "row_address_type": 0,
"multiplexing": 0, "multiplexing": 0,
"panel_type": "" "panel_type": ""
@@ -129,6 +130,9 @@
"plugin_rotation_order": [], "plugin_rotation_order": [],
"use_short_date_format": true, "use_short_date_format": true,
"vegas_scroll": { "vegas_scroll": {
"live_in_ticker": false,
"live_weight": 3,
"favorite_live_weight": 5,
"enabled": false, "enabled": false,
"scroll_speed": 50, "scroll_speed": 50,
"separator_width": 32, "separator_width": 32,
+89 -1
View File
@@ -64,10 +64,98 @@ JSON is optional.
| `target_fps` | `125` | Target frame rate | | `target_fps` | `125` | Target frame rate |
| `buffer_ahead` | `2` | Number of plugins buffered ahead | | `buffer_ahead` | `2` | Number of plugins buffered ahead |
This table is a subset — `display.vegas_scroll` supports 26 keys in This table is a subset — `display.vegas_scroll` supports 30 keys in
total. See the full list in total. See the full list in
[CONFIG_REFERENCE.md](CONFIG_REFERENCE.md#displayvegas_scroll--continuous-scroll-mode). [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md#displayvegas_scroll--continuous-scroll-mode).
### Live Content in the Ticker
By default, live content **preempts** Vegas mode: while any plugin reports
live priority, the display controller refuses to run the ticker and shows
that plugin's full-screen display instead. You get a big readable scoreboard,
but the marquee stops entirely for the duration of the game.
Set `live_in_ticker` to keep the ticker running and let live content take
**extra turns inside it** instead:
```json
"vegas_scroll": {
"live_in_ticker": true,
"live_weight": 3,
"favorite_live_weight": 5
}
```
#### Why weights exist
The rotation is otherwise a strict round robin — every plugin appears exactly
once per cycle. With a dozen plugins enabled, a live score comes round once a
lap and can be minutes old by the time you see it. A weight of *N* gives a
plugin *N* slots per cycle.
The slots are placed by **Smooth Weighted Round-Robin**, the same scheduler
the sports plugins use internally to rotate their own games. The important
property is that repeats are *spread through the cycle* rather than clumped:
three appearances in a row followed by a long silence would be worse than not
boosting at all.
Twelve plugins, with a favorite's baseball game and an ordinary live hockey
game (`live_weight: 3`, `favorite_live_weight: 5`):
```
baseball > hockey > weather > clock > baseball
stocks > news > flights > baseball > hockey
calendar > f1 > music > baseball > tides
birds > hockey > baseball
```
18 slots for 12 plugins. Baseball appears 5 times, hockey 3, everything else
once, and no plugin ever appears twice in a row — **including across the seam**
where the cycle loops back on itself. Smooth Weighted Round-Robin schedules the
heaviest item first and usually last as well, so the strip would otherwise show
it twice running at exactly the one join a within-cycle check cannot see. The
trailing repeat is moved into the widest remaining gap. Where a double is
unavoidable — a plugin holding most of the slots has to neighbour itself — the
schedule is left as it is.
#### Where the weight comes from
For each plugin in the rotation, in order:
1. **The plugin's own answer.** If it implements
`get_vegas_priority_weight()` and returns a number, that wins. This is the
only route for favorite-team awareness — the core can see *that* a game is
live, but not *whose*, so a scoreboard has to say so itself.
2. **The core's default.** When the plugin returns `None` (the base-class
default), a plugin where both `has_live_priority()` and `has_live_content()`
are true gets `live_weight`.
3. **Everything else** gets 1.
Because of step 2, **existing plugins need no changes** — any scoreboard with
`live_priority` enabled already gets extra turns. Step 1 is opt-in, for
plugins that want to distinguish a favorite's game from any other live game.
Weights are clamped to 110. A weight of 1 is no boost; a weight below 1 would
drop the plugin from the rotation entirely, which is never what is meant.
#### Things worth knowing
- **Weights are per plugin, not per game.** A scoreboard showing four live
games still occupies one slot at a time, rotating its own games within that
slot using its own `favorite_live_boost`. This controls how often the
*plugin* comes round.
- **The ticker is zero-sum.** Giving baseball 5 slots does not make the cycle
faster; it makes the cycle *longer* and everything else proportionally
rarer. If you want live scores sooner in wall-clock terms, pair this with a
smaller `plugins_per_cycle`.
- **Frequency is not freshness.** Each appearance redraws from the plugin's
current data (`refresh_updated_plugins()` drops cached content when a
plugin's data changes), but how current that data is depends on the
plugin's own `live_update_interval`. Showing a stale score five times a lap
is no better than showing it once.
- **Everything still appears.** A boost never starves another plugin out of
the cycle; low-weight plugins keep their single slot.
### Per-Plugin Configuration ### Per-Plugin Configuration
Override Vegas behavior for specific plugins: Override Vegas behavior for specific plugins:
+6 -1
View File
@@ -66,6 +66,7 @@ in `DisplayManager` (`src/display_manager.py`, ~lines 270295).
| `led_rgb_sequence` | string, `"RGB"` | | `led_rgb_sequence` | string, `"RGB"` |
| `limit_refresh_rate_hz` | int, `100` (code default 90) | | `limit_refresh_rate_hz` | int, `100` (code default 90) |
| `pixel_mapper_config` | string, `""` — e.g. `"U-mapper"` / `"Rotate:90"` | | `pixel_mapper_config` | string, `""` — e.g. `"U-mapper"` / `"Rotate:90"` |
| `orientation` | string, `"normal"``"180"` rotates the rendered image 180° for panels physically mounted upside down (e.g. to move the Pi/wiring to a more convenient side); composed onto `pixel_mapper_config` as a trailing `Rotate:180` mapper, so it stays independent of any custom `pixel_mapper_config` value |
| `row_address_type` | int, `0` — non-standard panel row addressing | | `row_address_type` | int, `0` — non-standard panel row addressing |
| `multiplexing` | int, `0` — panel multiplexing scheme | | `multiplexing` | int, `0` — panel multiplexing scheme |
| `panel_type` | string, `""` — set to `"FM6126A"` or `"FM6127"` for panels needing init | | `panel_type` | string, `""` — set to `"FM6126A"` or `"FM6127"` for panels needing init |
@@ -103,7 +104,8 @@ logical image to multiple chained physical panels.
## `display.vegas_scroll` — continuous scroll mode ## `display.vegas_scroll` — continuous scroll mode
Read by `src/vegas_mode/config.py` (`VegasScrollConfig.from_config`). See Read by `src/vegas_mode/config.py` (`VegasScrollConfig.from_config`). See
[ADVANCED_FEATURES.md](ADVANCED_FEATURES.md) for behavior details. [ADVANCED_FEATURES.md](ADVANCED_FEATURES.md) for behavior details, including
[live content in the ticker](ADVANCED_FEATURES.md#live-content-in-the-ticker).
| Key | Type / default | | Key | Type / default |
|---|---| |---|---|
@@ -134,6 +136,9 @@ Read by `src/vegas_mode/config.py` (`VegasScrollConfig.from_config`). See
| `max_cycle_duration` | int, `240` | | `max_cycle_duration` | int, `240` |
| `frame_based_scrolling` | bool, `true` — frame-count-based scroll stepping | | `frame_based_scrolling` | bool, `true` — frame-count-based scroll stepping |
| `scroll_delay` | float, `0.02` — seconds between scroll updates (~50 FPS) | | `scroll_delay` | float, `0.02` — seconds between scroll updates (~50 FPS) |
| `live_in_ticker` | bool, `false` — keep scrolling during live games instead of handing the display to a full-screen scoreboard |
| `live_weight` | int, `3` (110) — slots per cycle for a plugin with live content |
| `favorite_live_weight` | int, `5` (110) — slots per cycle when a plugin reports a favorite team is live |
## `sync` — multi-display synchronization ## `sync` — multi-display synchronization
+41
View File
@@ -170,6 +170,47 @@ Default returns `False`.
List of display modes to show during a live takeover. Default returns the List of display modes to show during a live takeover. Default returns the
plugin's `display_modes` from its manifest. plugin's `display_modes` from its manifest.
#### `get_vegas_priority_weight() -> Optional[int]`
How many slots per Vegas cycle this plugin should get. Default returns
`None`, which defers to the core.
The Vegas ticker is otherwise a strict round robin — every plugin appears
exactly once per cycle — so with a dozen plugins enabled a live score can be
minutes stale by the time it comes round. A weight of *N* gives the plugin
*N* slots per cycle, spread evenly through it rather than clumped.
**You usually do not need this.** When the hook returns `None`, the core
already gives a plugin `vegas_scroll.live_weight` whenever
`has_live_priority()` and `has_live_content()` are both true. Live sports get
extra turns with no code at all.
Implement it only when the plugin knows something the core cannot. The
motivating case is favorite teams — the core can see *that* a game is live,
but not *whose*:
```python
def get_vegas_priority_weight(self):
if not (self.has_live_priority() and self.has_live_content()):
return None # let the core decide
vegas = self.global_config.get('display', {}).get('vegas_scroll', {})
if self._favorite_is_live():
return vegas.get('favorite_live_weight', 5)
return vegas.get('live_weight', 3)
```
The weight is per *plugin*, not per game: a scoreboard showing four live games
still occupies one slot at a time and rotates its own games within it. Values
are clamped to 110 by the caller. An exception here is caught and logged, and
the core then falls back to its own live-content check — so a plugin whose
weight calculation is broken still gets `live_weight` for a game that really
is live, rather than being demoted to 1.
Only consulted when the user has set `vegas_scroll.live_in_ticker`. With the
default (`false`) live content preempts Vegas entirely and there is no ticker
to be weighted within. See
[ADVANCED_FEATURES.md](ADVANCED_FEATURES.md#live-content-in-the-ticker).
### Vegas scroll hooks ### Vegas scroll hooks
Vegas mode shows multiple plugins as a single continuous scroll instead of Vegas mode shows multiple plugins as a single continuous scroll instead of
+54 -3
View File
@@ -12,6 +12,8 @@ Follows LEDMatrix configuration management patterns:
""" """
import logging import logging
import time
import requests import requests
import json import json
from typing import Dict, Any, Optional, List from typing import Dict, Any, Optional, List
@@ -43,9 +45,34 @@ class BaseOddsManager:
self.logger = logging.getLogger(__name__) self.logger = logging.getLogger(__name__)
self.base_url = "https://sports.core.api.espn.com/v2/sports" self.base_url = "https://sports.core.api.espn.com/v2/sports"
# This path used a bare requests.get, so it identified itself as
# python-requests/x.y -- the one thing ESPN is known to reject. Around
# 2026-08-04 it began 403ing browser strings and bare custom tokens
# alike; what it accepts is a token with a URL that says who is
# calling. Every other ESPN caller in the tree already sends this
# (src/common/api_helper.py, src/base_classes/data_sources.py); the
# odds path was simply missed, and it is the one whose failures cost
# the caller its whole update budget.
#
# Deliberately no retry adapter, unlike api_helper: retries multiply
# request_timeout, which is set to 5s precisely to stay inside that
# budget. One try, then the cooldown below.
self.session = requests.Session()
self.session.headers.update({
'User-Agent': 'LEDMatrix/1.0 (+https://github.com/ChuckBuilds/LEDMatrix)',
'Accept': 'application/json',
})
# Configuration with defaults # Configuration with defaults
self.update_interval = 3600 # 1 hour default self.update_interval = 3600 # 1 hour default
self.request_timeout = 30 # 30 seconds default # Well under the plugin executor's 30s operation budget. At 30s a
# single stalled ESPN request consumed the entire budget and the whole
# update() was killed -- and odds are fetched per live game, inside the
# live update loop, with show_odds defaulting on. Losing one game's
# odds beats losing the update that carries every game's score.
self.request_timeout = 5
# Set when a request fails; until then, skip the network entirely.
self._skip_network_until = 0.0
self.cache_ttl = 1800 # 30 minutes default self.cache_ttl = 1800 # 30 minutes default
# Load configuration if available # Load configuration if available
@@ -73,6 +100,14 @@ class BaseOddsManager:
except Exception as e: except Exception as e:
self.logger.warning(f"Failed to load BaseOddsManager configuration: {e}") self.logger.warning(f"Failed to load BaseOddsManager configuration: {e}")
# After a network failure, stop trying for this long and serve cache only.
# A short per-request timeout bounds one stall, but a full Sunday slate is
# ~16 games fetched in a loop, so 16 consecutive timeouts still blow the
# budget. When ESPN is unreachable it is unreachable for all of them, so
# the first failure is enough to know: skip the rest of this pass and try
# again shortly.
_FAILURE_COOLDOWN = 60.0
def get_odds(self, sport: str | None, league: str | None, event_id: str, def get_odds(self, sport: str | None, league: str | None, event_id: str,
update_interval_seconds: int = None) -> Optional[Dict[str, Any]]: update_interval_seconds: int = None) -> Optional[Dict[str, Any]]:
""" """
@@ -101,6 +136,16 @@ class BaseOddsManager:
self.logger.info(f"Using cached odds from ESPN for {cache_key}") self.logger.info(f"Using cached odds from ESPN for {cache_key}")
return cached_data return cached_data
if time.monotonic() < self._skip_network_until:
# A recent request failed, so ESPN is very likely still unreachable.
# Returning now keeps the caller's update inside its time budget
# instead of paying the timeout again for every remaining game.
self.logger.debug(
"Skipping odds fetch for %s: a recent request failed, holding off "
"for another %.0fs", cache_key,
self._skip_network_until - time.monotonic())
return None
self.logger.info(f"Cache miss - fetching fresh odds from ESPN for {cache_key}") self.logger.info(f"Cache miss - fetching fresh odds from ESPN for {cache_key}")
try: try:
@@ -117,10 +162,12 @@ class BaseOddsManager:
url = f"{self.base_url}/{sport}/leagues/{espn_league}/events/{event_id}/competitions/{event_id}/odds" url = f"{self.base_url}/{sport}/leagues/{espn_league}/events/{event_id}/competitions/{event_id}/odds"
self.logger.info(f"Requesting odds from URL: {url}") self.logger.info(f"Requesting odds from URL: {url}")
response = requests.get(url, timeout=self.request_timeout) response = self.session.get(url, timeout=self.request_timeout)
response.raise_for_status() response.raise_for_status()
raw_data = response.json() raw_data = response.json()
self._skip_network_until = 0.0 # reachable again
self.logger.debug(f"Received raw odds data from ESPN: {json.dumps(raw_data, indent=2)}") self.logger.debug(f"Received raw odds data from ESPN: {json.dumps(raw_data, indent=2)}")
odds_data = self._extract_espn_data(raw_data) odds_data = self._extract_espn_data(raw_data)
@@ -140,7 +187,11 @@ class BaseOddsManager:
return odds_data return odds_data
except requests.exceptions.RequestException as e: except requests.exceptions.RequestException as e:
self.logger.error(f"Error fetching odds from ESPN API for {cache_key}: {e}") self._skip_network_until = time.monotonic() + self._FAILURE_COOLDOWN
self.logger.error(
"Error fetching odds from ESPN API for %s: %s. Holding off on odds "
"for %.0fs so a slate of games does not pay this timeout each.",
cache_key, e, self._FAILURE_COOLDOWN)
except json.JSONDecodeError: except json.JSONDecodeError:
self.logger.error(f"Error decoding JSON response from ESPN API for {cache_key}.") self.logger.error(f"Error decoding JSON response from ESPN API for {cache_key}.")
+80 -1
View File
@@ -14,6 +14,13 @@ import zlib
from typing import Dict, Any, Optional, Protocol from typing import Dict, Any, Optional, Protocol
from datetime import datetime from datetime import datetime
# How old an abandoned write's temp file must be before the sweep removes it.
# A real write holds its temp file for milliseconds, so an hour is far beyond
# any in-flight write while still clearing the same day's debris. Deliberately
# not tied to the retention policies: those describe how long data stays
# useful, and a half-written file was never useful.
_ORPHAN_TEMP_MAX_AGE_SECONDS = 3600
class CacheStrategyProtocol(Protocol): class CacheStrategyProtocol(Protocol):
@@ -112,6 +119,22 @@ class DiskCache:
record_ts = None record_ts = None
now = time.time() now = time.time()
# An explicit per-entry ttl wins over the caller's max_age. The
# caller that wrote the record knows what its data is; max_age is
# inferred from substrings in the key ("live", "odds", "stock") and
# is only a fallback for records that never said. Until now the ttl
# was stored and ignored, so `set(key, data, ttl=...)` did nothing
# at all -- 48 plugin call sites and 4 in the core were writing a
# number no read path consulted.
effective_max_age = max_age
if isinstance(record, dict):
stored_ttl = record.get('ttl')
if isinstance(stored_ttl, (int, float)) and not isinstance(stored_ttl, bool) \
and stored_ttl >= 0:
effective_max_age = stored_ttl
max_age = effective_max_age
# max_age=None means "never expires" (mirrors MemoryCache and the # max_age=None means "never expires" (mirrors MemoryCache and the
# cache_manager docstring). Guard it explicitly — otherwise the # cache_manager docstring). Guard it explicitly — otherwise the
# comparison below raises TypeError and the record is treated as a # comparison below raises TypeError and the record is treated as a
@@ -331,6 +354,23 @@ class DiskCache:
"""Get the cache directory path.""" """Get the cache directory path."""
return self.cache_dir return self.cache_dir
@staticmethod
def _is_orphaned_temp(filename: str) -> bool:
"""Whether a name is one of set()'s temp files rather than real data.
Matches only what this class creates: mkstemp with a prefix of
".<cache filename>." , so ".weather.json.a1b2c3d4". The shape is
checked rather than just the leading dot, because this predicate
deletes things -- a stray dotfile someone left in the cache directory
is not ours to remove, and a completed ".json" never is either.
"""
if not filename.startswith('.') or filename.endswith('.json'):
return False
head, sep, suffix = filename.rpartition('.json.')
# head is the key (non-empty after the leading dot), suffix is
# mkstemp's random component.
return bool(sep) and len(head) > 1 and bool(suffix)
def cleanup_expired_files(self, cache_strategy: CacheStrategyProtocol, retention_policies: Dict[str, int]) -> Dict[str, Any]: def cleanup_expired_files(self, cache_strategy: CacheStrategyProtocol, retention_policies: Dict[str, int]) -> Dict[str, Any]:
""" """
Clean up expired cache files based on retention policies. Clean up expired cache files based on retention policies.
@@ -365,12 +405,51 @@ class DiskCache:
try: try:
with self._lock: with self._lock:
# Get snapshot of files while holding lock briefly # Get snapshot of files while holding lock briefly
filenames = [f for f in os.listdir(self.cache_dir) if f.endswith('.json')] entries = os.listdir(self.cache_dir)
except OSError as list_error: except OSError as list_error:
self.logger.error("Error listing cache directory %s: %s", self.cache_dir, list_error, exc_info=True) self.logger.error("Error listing cache directory %s: %s", self.cache_dir, list_error, exc_info=True)
stats['errors'] += 1 stats['errors'] += 1
return stats return stats
filenames = [f for f in entries if f.endswith('.json')]
# Sweep temp files abandoned by a write that never finished. set()
# removes its own in a finally, so these are the ones where the
# process died between mkstemp and os.replace -- a SIGKILL, a lost
# restart race, a power cut. Nothing ever collected them: they are
# named ".<key>.json.<random>", and the scan above only matches
# names ending in .json, so they accumulated indefinitely. Measured
# on a live rig: 76 files, 1,050 MB, 81% of the whole cache
# directory, the oldest six months old.
stats['orphan_temp_files_deleted'] = 0
for filename in (f for f in entries if self._is_orphaned_temp(f)):
# Counted as scanned like any other candidate, so files_deleted
# can never exceed files_scanned and the summary line reads
# honestly ("77/8864", not "77/0").
stats['files_scanned'] += 1
path = os.path.join(self.cache_dir, filename)
try:
# An in-flight write lives for milliseconds, so anything
# this old is certainly abandoned rather than in progress.
if (current_time - os.path.getmtime(path)) <= _ORPHAN_TEMP_MAX_AGE_SECONDS:
continue
with self._lock:
size = os.path.getsize(path)
os.remove(path)
stats['files_deleted'] += 1
stats['orphan_temp_files_deleted'] += 1
stats['space_freed_bytes'] += size
except FileNotFoundError:
continue # another sweep got there first
except OSError as e:
stats['errors'] += 1
self.logger.warning("Error deleting orphaned temp file %s: %s", filename, e)
if stats['orphan_temp_files_deleted']:
self.logger.info(
"Removed %d abandoned cache temp file(s)",
stats['orphan_temp_files_deleted'])
# Process files outside the lock to avoid blocking get/set operations # Process files outside the lock to avoid blocking get/set operations
for filename in filenames: for filename in filenames:
stats['files_scanned'] += 1 stats['files_scanned'] += 1
+10
View File
@@ -57,6 +57,16 @@ class MemoryCache:
if timestamp is None: if timestamp is None:
return None return None
# An explicit per-entry ttl wins over the caller's max_age, matching
# DiskCache. max_age is inferred from substrings in the key and is
# only a fallback for records that did not say what they wanted.
record = self._cache[key]
if isinstance(record, dict):
stored_ttl = record.get('ttl')
if isinstance(stored_ttl, (int, float)) and not isinstance(stored_ttl, bool) \
and stored_ttl >= 0:
max_age = stored_ttl
# Check expiration # Check expiration
if max_age is not None and (now - timestamp) > max_age: if max_age is not None and (now - timestamp) > max_age:
# Expired - remove it # Expired - remove it
+44 -3
View File
@@ -47,6 +47,20 @@ from src.cache.disk_cache import DateTimeEncoder # noqa: F401 - deliberate re-e
class CacheManager: class CacheManager:
"""Manages caching of API responses to reduce API calls.""" """Manages caching of API responses to reduce API calls."""
# Which cache directories already have a cleanup thread in this process.
#
# The sweep is directory-scoped work -- it lists a directory and deletes
# from it -- so one per directory is the right number no matter how many
# managers exist. Nothing enforced that before: every instance started its
# own, and because the loop closes over `self`, a discarded manager could
# never be collected and its thread woke to re-scan the same directory
# every 24 hours for the life of the process. Startup validation runs
# twice and built a throwaway manager each time, so a display process
# carried three threads for one cache.
_cleanup_owners: Dict[str, 'CacheManager'] = {}
_cleanup_owners_lock = threading.Lock()
def __init__(self) -> None: def __init__(self) -> None:
# Initialize logger first # Initialize logger first
self.logger: logging.Logger = get_logger(__name__) self.logger: logging.Logger = get_logger(__name__)
@@ -594,8 +608,10 @@ class CacheManager:
Args: Args:
key: Cache key key: Cache key
data: Data to cache data: Data to cache
ttl: Optional time-to-live in seconds (stored for compatibility but ttl: Time-to-live in seconds for this entry. Takes precedence over
expiration is still controlled via max_age when reading) the max_age a reader would otherwise apply, which is inferred
from the key and is only a fallback for entries that did not
say. Omit it to keep that inferred behaviour.
""" """
cache_data = { cache_data = {
'data': data, 'data': data,
@@ -716,11 +732,29 @@ class CacheManager:
} }
def start_cleanup_thread(self) -> None: def start_cleanup_thread(self) -> None:
"""Start background thread for periodic disk cache cleanup.""" """Start background thread for periodic disk cache cleanup.
At most one thread per cache directory per process: the sweep is
directory-scoped, so a second one only duplicates the scan.
"""
if self._cleanup_thread and self._cleanup_thread.is_alive(): if self._cleanup_thread and self._cleanup_thread.is_alive():
self.logger.debug("Cleanup thread already running") self.logger.debug("Cleanup thread already running")
return return
with CacheManager._cleanup_owners_lock:
owner = CacheManager._cleanup_owners.get(self.cache_dir)
if owner is not None and owner is not self:
thread = owner._cleanup_thread
if thread is not None and thread.is_alive():
self.logger.debug(
"Cleanup thread for %s already owned by another cache "
"manager in this process; not starting a second",
self.cache_dir)
return
# The owner's thread died or was stopped -- take over.
CacheManager._cleanup_owners[self.cache_dir] = self
def cleanup_loop(): def cleanup_loop():
"""Background loop that runs cleanup periodically.""" """Background loop that runs cleanup periodically."""
self.logger.info("Disk cache cleanup thread started (interval: %d hours)", self.logger.info("Disk cache cleanup thread started (interval: %d hours)",
@@ -768,6 +802,13 @@ class CacheManager:
Signals the thread to stop and waits for it to finish (with timeout). Signals the thread to stop and waits for it to finish (with timeout).
This allows for clean shutdown during testing or application termination. This allows for clean shutdown during testing or application termination.
""" """
# Release ownership first and unconditionally, so a manager that never
# started a thread (or whose thread already exited) cannot keep the
# directory claimed and block a live manager from sweeping it.
with CacheManager._cleanup_owners_lock:
if CacheManager._cleanup_owners.get(self.cache_dir) is self:
del CacheManager._cleanup_owners[self.cache_dir]
if not self._cleanup_thread or not self._cleanup_thread.is_alive(): if not self._cleanup_thread or not self._cleanup_thread.is_alive():
self.logger.debug("Cleanup thread not running") self.logger.debug("Cleanup thread not running")
return return
+80 -8
View File
@@ -44,6 +44,20 @@ from src.common.sync_manager import DisplaySyncManager, SyncRole
# Get logger with consistent configuration # Get logger with consistent configuration
logger = get_logger(__name__) logger = get_logger(__name__)
# How long startup will wait for plugins to fetch their first data before
# showing anything. Each plugin's update blocks for up to the executor's 30s
# timeout and they run one after another, so the uncapped total is the sum of
# every slow plugin: 82 seconds on the worst boot measured, with a blank panel
# throughout. Whatever does not finish in time is picked up by the scheduled
# update tick moments later, with the display already running.
_INITIAL_UPDATE_BUDGET_SECONDS = 20.0
# The least budget worth starting a plugin with. Below this the plugin is
# deferred instead: granting it a floor would let the pass run past its
# deadline, and granting it the true remainder would record a timeout for a
# slot it never had a chance to use.
_MIN_INITIAL_UPDATE_TIMEOUT_SECONDS = 2.0
# Vegas mode import (lazy loaded to avoid circular imports) # Vegas mode import (lazy loaded to avoid circular imports)
_vegas_mode_imported = False _vegas_mode_imported = False
VegasModeCoordinator = None VegasModeCoordinator = None
@@ -90,7 +104,8 @@ class DisplayController:
# Validate startup configuration # Validate startup configuration
try: try:
from src.startup_validator import StartupValidator from src.startup_validator import StartupValidator
validator = StartupValidator(self.config_manager) validator = StartupValidator(self.config_manager,
cache_manager=self.cache_manager)
is_valid, errors, warnings = validator.validate_all() is_valid, errors, warnings = validator.validate_all()
if warnings: if warnings:
@@ -258,7 +273,8 @@ class DisplayController:
# Validate plugins after plugin manager is created # Validate plugins after plugin manager is created
try: try:
from src.startup_validator import StartupValidator from src.startup_validator import StartupValidator
validator = StartupValidator(self.config_manager, self.plugin_manager) validator = StartupValidator(self.config_manager, self.plugin_manager,
cache_manager=self.cache_manager)
is_valid, errors, warnings = validator.validate_all() is_valid, errors, warnings = validator.validate_all()
if warnings: if warnings:
@@ -461,7 +477,7 @@ class DisplayController:
# Initial data update for plugins (ensures data available on first display) # Initial data update for plugins (ensures data available on first display)
logger.info("Performing initial plugin data update...") logger.info("Performing initial plugin data update...")
update_start = time.time() update_start = time.time()
self._update_modules() self._update_modules(deadline=update_start + _INITIAL_UPDATE_BUDGET_SECONDS)
logger.info("Initial plugin update completed in %.3f seconds", time.time() - update_start) logger.info("Initial plugin update completed in %.3f seconds", time.time() - update_start)
# Initialize Vegas mode coordinator # Initialize Vegas mode coordinator
@@ -817,14 +833,42 @@ class DisplayController:
self._cached_target_brightness = normal_brightness # persist for minute-gate self._cached_target_brightness = normal_brightness # persist for minute-gate
return normal_brightness return normal_brightness
def _update_modules(self): def _update_modules(self, deadline: Optional[float] = None):
"""Update all plugin modules.""" """Update all plugin modules.
Args:
deadline: Wall-clock time after which remaining plugins are left
for the scheduled update tick instead of being waited on. Each
update blocks this thread for up to the executor's timeout, and
they run one after another, so without a bound the total is the
sum of every slow plugin on the system. Measured at startup on
a live rig: 82 seconds, 55 and 26 on the two boots before -- all
of it with nothing on the panel.
"""
if not self.plugin_manager: if not self.plugin_manager:
return return
# Update all loaded plugins # Update all loaded plugins
plugins_dict = getattr(self.plugin_manager, 'loaded_plugins', None) or getattr(self.plugin_manager, 'plugins', {}) plugins_dict = getattr(self.plugin_manager, 'loaded_plugins', None) or getattr(self.plugin_manager, 'plugins', {})
deferred = []
for plugin_id, plugin_instance in plugins_dict.items(): for plugin_id, plugin_instance in plugins_dict.items():
update_timeout = None
if deadline is not None:
update_timeout = deadline - time.time()
if update_timeout < _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS:
# Too little left to be worth starting. Deferring rather
# than granting a floor keeps the budget a real ceiling --
# clamping up to a minimum let a plugin that began with a
# sliver left run on past the deadline -- and a plugin
# handed a slot it cannot use would just be recorded as
# having timed out.
#
# Nothing is lost either way: a plugin that has never
# updated is immediately due, so run_scheduled_updates()
# picks it up within seconds, with the display already
# running.
deferred.append(plugin_id)
continue
# Check circuit breaker before attempting update # Check circuit breaker before attempting update
if hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker: if hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
if self.plugin_manager.health_tracker.should_skip_plugin(plugin_id): if self.plugin_manager.health_tracker.should_skip_plugin(plugin_id):
@@ -833,7 +877,13 @@ class DisplayController:
# Use PluginExecutor if available for safe execution # Use PluginExecutor if available for safe execution
if hasattr(self.plugin_manager, 'plugin_executor'): if hasattr(self.plugin_manager, 'plugin_executor'):
success = self.plugin_manager.plugin_executor.execute_update(plugin_instance, plugin_id) # The remaining budget is the timeout, so the pass cannot
# run past its deadline. Bounding the loop alone did not do
# it: the last plugin to start could still block for the
# executor's full 30s, which turned a 20s budget into a 31.8s
# pass on the rig.
success = self.plugin_manager.plugin_executor.execute_update(
plugin_instance, plugin_id, timeout=update_timeout)
if success and hasattr(self.plugin_manager, 'plugin_last_update'): if success and hasattr(self.plugin_manager, 'plugin_last_update'):
self.plugin_manager.plugin_last_update[plugin_id] = time.time() self.plugin_manager.plugin_last_update[plugin_id] = time.time()
else: else:
@@ -852,6 +902,12 @@ class DisplayController:
if hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker: if hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
self.plugin_manager.health_tracker.record_failure(plugin_id, exc) self.plugin_manager.health_tracker.record_failure(plugin_id, exc)
if deferred:
logger.info(
"Initial update budget spent; %d plugin(s) left to the update "
"tick so the display can start: %s",
len(deferred), ", ".join(deferred))
def _tick_plugin_updates_for_vegas(self) -> None: def _tick_plugin_updates_for_vegas(self) -> None:
"""Run scheduled plugin updates and tell Vegas mode which plugins """Run scheduled plugin updates and tell Vegas mode which plugins
actually got fresh data, so it can hot-swap them into the scroll actually got fresh data, so it can hot-swap them into the scroll
@@ -1638,6 +1694,12 @@ class DisplayController:
logger.warning("Error checking live priority for %s: %s", mode_name, e) logger.warning("Error checking live priority for %s: %s", mode_name, e)
return live return live
def _vegas_keeps_live_in_ticker(self) -> bool:
"""Whether live content should stay in the ticker instead of preempting it."""
coordinator = getattr(self, 'vegas_coordinator', None)
config = getattr(coordinator, 'vegas_config', None)
return bool(getattr(config, 'live_in_ticker', False))
def _check_live_priority(self, advance=False): def _check_live_priority(self, advance=False):
"""Return the live-priority mode to display, or None if nothing is live. """Return the live-priority mode to display, or None if nothing is live.
@@ -1851,14 +1913,24 @@ class DisplayController:
# Check for live priority content and switch to it immediately. # Check for live priority content and switch to it immediately.
# advance=True so multiple simultaneously-live games take turns # advance=True so multiple simultaneously-live games take turns
# (round-robin) instead of pinning to the first plugin. # (round-robin) instead of pinning to the first plugin.
if not self.on_demand_active and not wifi_status_data: # Skipped when the ticker is keeping live content: switching
# the rotation underneath Vegas would move current_mode_index
# and stash a resume point for a takeover that never happens.
if (not self.on_demand_active and not wifi_status_data
and not (self._is_vegas_mode_active()
and self._vegas_keeps_live_in_ticker())):
live_priority_mode = self._check_live_priority(advance=True) live_priority_mode = self._check_live_priority(advance=True)
self._apply_live_priority(live_priority_mode) self._apply_live_priority(live_priority_mode)
# Vegas scroll mode - continuous ticker across all plugins # Vegas scroll mode - continuous ticker across all plugins
# Priority: on-demand > wifi-status > live-priority > vegas > normal rotation # Priority: on-demand > wifi-status > live-priority > vegas > normal rotation
if self._is_vegas_mode_active() and not wifi_status_data: if self._is_vegas_mode_active() and not wifi_status_data:
live_mode = self._check_live_priority() # Live content normally preempts the ticker entirely. With
# vegas_scroll.live_in_ticker the marquee keeps running and
# the live plugin takes extra turns inside it instead --
# see StreamManager._apply_priority_weights.
live_mode = (None if self._vegas_keeps_live_in_ticker()
else self._check_live_priority())
if not live_mode: if not live_mode:
try: try:
# Run Vegas mode iteration # Run Vegas mode iteration
+112 -3
View File
@@ -25,6 +25,7 @@ the same object.
import json import json
import os import os
import socket
import tempfile import tempfile
if os.getenv("EMULATOR", "false") == "true": if os.getenv("EMULATOR", "false") == "true":
from RGBMatrixEmulator import RGBMatrix, RGBMatrixOptions from RGBMatrixEmulator import RGBMatrix, RGBMatrixOptions
@@ -258,6 +259,26 @@ class DisplayManager:
# Initialize managers # Initialize managers
# Calendar manager is now initialized by DisplayController # Calendar manager is now initialized by DisplayController
# Orientation setting -> rpi-rgb-led-matrix "Rotate:<deg>" pixel-mapper suffix.
# "normal" needs no suffix since 0 degrees is the identity transform.
_ORIENTATION_ROTATE_DEGREES = {'normal': None, '90': 90, '180': 180, '270': 270}
def _build_pixel_mapper_config(self, hardware_config: dict) -> str:
"""Compose the raw pixel_mapper_config string with the orientation setting.
`pixel_mapper_config` stays available as a free-form advanced field (e.g.
for "U-mapper" chain layouts); `orientation` is the user-facing dropdown
for physical mounting (e.g. panels mounted upside down) and is appended as
a "Rotate:<deg>" mapper rather than overwriting any existing config.
"""
base_mapper = (hardware_config.get('pixel_mapper_config') or '').strip()
orientation = hardware_config.get('orientation', 'normal')
degrees = self._ORIENTATION_ROTATE_DEGREES.get(orientation)
if degrees is None:
return base_mapper
rotate_mapper = f'Rotate:{degrees}'
return f'{base_mapper};{rotate_mapper}' if base_mapper else rotate_mapper
def _setup_matrix(self): def _setup_matrix(self):
"""Initialize the RGB matrix with configuration settings.""" """Initialize the RGB matrix with configuration settings."""
_init_error_str = None _init_error_str = None
@@ -283,7 +304,7 @@ class DisplayManager:
options.pwm_bits = hardware_config.get('pwm_bits', 10) options.pwm_bits = hardware_config.get('pwm_bits', 10)
options.pwm_lsb_nanoseconds = hardware_config.get('pwm_lsb_nanoseconds', 150) options.pwm_lsb_nanoseconds = hardware_config.get('pwm_lsb_nanoseconds', 150)
options.led_rgb_sequence = hardware_config.get('led_rgb_sequence', 'RGB') options.led_rgb_sequence = hardware_config.get('led_rgb_sequence', 'RGB')
options.pixel_mapper_config = hardware_config.get('pixel_mapper_config', '') options.pixel_mapper_config = self._build_pixel_mapper_config(hardware_config)
options.row_address_type = hardware_config.get('row_address_type', 0) options.row_address_type = hardware_config.get('row_address_type', 0)
options.multiplexing = hardware_config.get('multiplexing', 0) options.multiplexing = hardware_config.get('multiplexing', 0)
options.panel_type = hardware_config.get('panel_type', '') options.panel_type = hardware_config.get('panel_type', '')
@@ -497,6 +518,91 @@ class DisplayManager:
logger.warning(f"[BRIGHTNESS] Matrix does not support brightness property: {e}", exc_info=True) logger.warning(f"[BRIGHTNESS] Matrix does not support brightness property: {e}", exc_info=True)
return -1 return -1
@staticmethod
def _local_ip() -> Optional[str]:
"""This device's address on the network it routes through, or None.
Deliberately not `hostname -I` or a systemctl probe for AP mode, which
is how the web launcher does it: both spawn processes with multi-second
timeouts, and this runs on the startup path the rest of this change
exists to shorten. Connecting a UDP socket sends no packets -- it only
asks the kernel which source address it would use -- so it costs
microseconds and works with the network down, as long as a route
exists.
"""
sock = None
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.settimeout(0.2)
sock.connect(("8.8.8.8", 80)) # nosec B104 - no traffic; selects a route
ip = sock.getsockname()[0]
return ip if ip and not ip.startswith("127.") else None
except OSError:
return None
finally:
if sock is not None:
try:
sock.close()
except OSError:
pass
def _fitting_font(self, lines, width):
"""The largest font from the usual ladder that fits every line."""
candidates = [self.font,
("assets/fonts/4x6-font.ttf", 6)]
for candidate in candidates:
try:
font = candidate
if isinstance(candidate, tuple):
font = ImageFont.truetype(candidate[0], candidate[1])
if all(self.draw.textlength(t, font=font) <= width for t in lines):
return font
except (OSError, ValueError, AttributeError):
continue
return self.font
def _draw_startup_banner(self, lines, width: int, height: int) -> None:
"""Centre `lines` over whatever the test pattern already drew.
This screen stays on the panel for the whole initial plugin update, and
on a headless Pi it is the only place the device's address appears
without going looking for it -- so it has to be readable off a wall,
not merely present.
The font is chosen to fit rather than fixed at 8px: "Initializing" is
96px in PressStart2P, which ran off the side of a 64px panel even
before an address was added. And the pattern is punched out behind the
text, because the diagonal runs through the middle of the panel, which
is exactly where this sits.
The text stays blue. It is not decoration: the pattern draws one pure
channel per element -- red border, green diagonal, blue text -- so that
a glance at the panel says whether led_rgb_sequence is right. Swap the
wiring to BGR and the border comes up blue and this text red. Drawing
it white would light all three channels and destroy the only blue
reference on the screen, which is why it is worth a comment rather
than a quiet preference.
"""
if not lines:
return
font = self._fitting_font(lines, width - 2)
line_height = self.draw.textbbox((0, 0), "Ag", font=font)[3] + 1
block_height = line_height * len(lines)
block_top = max(1, (height - block_height) // 2)
block_width = max(self.draw.textlength(t, font=font) for t in lines)
block_left = max(0, (width - block_width) // 2)
self.draw.rectangle(
[block_left - 2, block_top - 1,
block_left + block_width + 1, block_top + block_height],
fill=(0, 0, 0))
for row, line in enumerate(lines):
line_width = self.draw.textlength(line, font=font)
self.draw.text(
(max(0, (width - line_width) // 2), block_top + row * line_height),
line, font=font, fill=(0, 0, 255))
def _draw_test_pattern(self): def _draw_test_pattern(self):
"""Draw a test pattern to verify the display is working.""" """Draw a test pattern to verify the display is working."""
try: try:
@@ -516,8 +622,11 @@ class DisplayManager:
# Draw a diagonal line # Draw a diagonal line
self.draw.line([0, 0, self.matrix.width-1, self.matrix.height-1], fill=(0, 255, 0)) self.draw.line([0, 0, self.matrix.width-1, self.matrix.height-1], fill=(0, 255, 0))
# Draw some text - changed from "TEST" to "Initializing" with smaller font lines = ["Initializing"]
self.draw.text((10, 10), "Initializing", font=self.font, fill=(0, 0, 255)) ip = self._local_ip()
if ip:
lines.append(ip)
self._draw_startup_banner(lines, self.matrix.width, self.matrix.height)
# Update the display once after everything is drawn # Update the display once after everything is drawn
self.update_display() self.update_display()
+42
View File
@@ -555,6 +555,48 @@ class BasePlugin(ABC):
""" """
return False return False
def get_vegas_priority_weight(self) -> Optional[int]:
"""How many slots per Vegas cycle this plugin should get, or None.
The Vegas ticker is otherwise a strict round robin: every plugin
appears exactly once per cycle. With a dozen plugins enabled that puts
minutes between a live score and its next appearance. A weight of N
gives the plugin N slots per cycle, spread evenly through it rather
than clumped together.
Return ``None`` (the default) to let the core decide. It gives a
plugin ``vegas_scroll.live_weight`` when ``has_live_priority()`` and
``has_live_content()`` are both true, and 1 otherwise -- so live sports
already get extra turns without implementing this at all.
Implement it only when the plugin knows something the core cannot. The
motivating case is favorite teams: the core can see *that* a game is
live but not *whose*, so a scoreboard that wants its favorite's game
shown more often than other live games has to say so::
def get_vegas_priority_weight(self):
if not (self.has_live_priority() and self.has_live_content()):
return None # let the core decide
cfg = self.global_config.get('display', {}).get('vegas_scroll', {})
if self._favorite_is_live():
return cfg.get('favorite_live_weight', 5)
return cfg.get('live_weight', 3)
The weight is per *plugin*, not per game. A scoreboard showing four
live games still occupies one slot at a time and rotates its own games
within that slot; this controls how often the plugin itself comes
round.
Raising is safe: the core logs it and falls back to its own
live-content check, so a broken weight calculation costs the plugin
the favorite distinction but not the live boost.
Returns:
Slots per cycle (clamped to 1..10 by the caller), or None to
defer to the core's own live-content weighting.
"""
return None
def get_live_modes(self) -> List[str]: def get_live_modes(self) -> List[str]:
""" """
Get list of display modes that should be used during live priority takeover. Get list of display modes that should be used during live priority takeover.
+20 -1
View File
@@ -15,16 +15,23 @@ from src.logging_config import get_logger
class StartupValidator: class StartupValidator:
"""Validates system state on startup.""" """Validates system state on startup."""
def __init__(self, config_manager: Any, plugin_manager: Optional[Any] = None) -> None: def __init__(self, config_manager: Any, plugin_manager: Optional[Any] = None,
cache_manager: Optional[Any] = None) -> None:
""" """
Initialize the startup validator. Initialize the startup validator.
Args: Args:
config_manager: ConfigManager instance config_manager: ConfigManager instance
plugin_manager: Optional PluginManager instance plugin_manager: Optional PluginManager instance
cache_manager: The CacheManager the application will actually use.
Pass it. Without one this validator builds its own just to read
a directory path, which reports on a cache the app does not
use and leaves behind a cleanup thread that nothing stops --
validation runs twice per startup, so that was two of them.
""" """
self.config_manager = config_manager self.config_manager = config_manager
self.plugin_manager = plugin_manager self.plugin_manager = plugin_manager
self.cache_manager = cache_manager
self.logger = get_logger(__name__) self.logger = get_logger(__name__)
self.errors: List[str] = [] self.errors: List[str] = []
self.warnings: List[str] = [] self.warnings: List[str] = []
@@ -91,8 +98,20 @@ class StartupValidator:
def _validate_cache_directory(self) -> None: def _validate_cache_directory(self) -> None:
"""Validate cache directory permissions.""" """Validate cache directory permissions."""
try: try:
cache_manager = self.cache_manager
if cache_manager is None:
# No caller supplied one (older embedders, direct use in a
# script). Build one, but do not leave its cleanup thread
# running behind us -- this instance is discarded on the next
# line but the thread is a closure over it, so it would never
# be collected.
from src.cache_manager import CacheManager from src.cache_manager import CacheManager
cache_manager = CacheManager() cache_manager = CacheManager()
try:
cache_dir = cache_manager.get_cache_dir()
finally:
cache_manager.stop_cleanup_thread()
else:
cache_dir = cache_manager.get_cache_dir() cache_dir = cache_manager.get_cache_dir()
if not cache_dir: if not cache_dir:
+44
View File
@@ -125,6 +125,32 @@ class VegasModeConfig:
plugin_order: List[str] = field(default_factory=list) plugin_order: List[str] = field(default_factory=list)
excluded_plugins: Set[str] = field(default_factory=set) excluded_plugins: Set[str] = field(default_factory=set)
# --- Live content in the ticker -------------------------------------
#
# By default a live game preempts Vegas entirely: the display controller
# refuses to run the ticker while any plugin reports live priority, and you
# get the full-screen scoreboard instead. Set live_in_ticker to keep the
# marquee running and let live content take extra turns within it.
#
# The rotation is otherwise a strict round robin -- every plugin appears
# exactly once per cycle -- so with a dozen plugins enabled a live score
# comes round once a lap and can be minutes old on screen. Weighting lets a
# plugin claim several slots per cycle instead.
#
# Weights are per plugin, not per game: a scoreboard showing four live
# games still occupies one slot at a time, and rotates its own games within
# that slot using its own favorite_live_boost.
live_in_ticker: bool = False
# Slots per cycle for a plugin reporting live content. 1 disables the boost
# and restores the plain round robin.
live_weight: int = 3
# Slots per cycle for a plugin whose live content involves a favorite team.
# Only plugins implementing get_vegas_priority_weight() can claim this --
# the core cannot tell whose game is on, so the plugin reports it.
favorite_live_weight: int = 5
# Performance settings # Performance settings
target_fps: int = 125 # Target frame rate target_fps: int = 125 # Target frame rate
buffer_ahead: int = 2 # Number of plugins to buffer ahead buffer_ahead: int = 2 # Number of plugins to buffer ahead
@@ -175,6 +201,12 @@ class VegasModeConfig:
overflow_mode=str(vegas_config.get('overflow_mode', 'rotate')), overflow_mode=str(vegas_config.get('overflow_mode', 'rotate')),
plugin_order=list(vegas_config.get('plugin_order', [])), plugin_order=list(vegas_config.get('plugin_order', [])),
excluded_plugins=set(vegas_config.get('excluded_plugins', [])), excluded_plugins=set(vegas_config.get('excluded_plugins', [])),
live_in_ticker=bool(vegas_config.get('live_in_ticker', False)),
# Clamped: a weight below 1 would drop the plugin from the rotation
# entirely, and a very large one starves everything else.
live_weight=max(1, min(10, int(vegas_config.get('live_weight', 3)))),
favorite_live_weight=max(
1, min(10, int(vegas_config.get('favorite_live_weight', 5)))),
target_fps=int(vegas_config.get('target_fps', 125)), target_fps=int(vegas_config.get('target_fps', 125)),
buffer_ahead=int(vegas_config.get('buffer_ahead', 2)), buffer_ahead=int(vegas_config.get('buffer_ahead', 2)),
frame_based_scrolling=vegas_config.get('frame_based_scrolling', True), frame_based_scrolling=vegas_config.get('frame_based_scrolling', True),
@@ -204,6 +236,9 @@ class VegasModeConfig:
'lead_in_width': self.lead_in_width, 'lead_in_width': self.lead_in_width,
'plugins_per_cycle': self.plugins_per_cycle, 'plugins_per_cycle': self.plugins_per_cycle,
'max_plugin_width_ratio': self.max_plugin_width_ratio, 'max_plugin_width_ratio': self.max_plugin_width_ratio,
'live_in_ticker': self.live_in_ticker,
'live_weight': self.live_weight,
'favorite_live_weight': self.favorite_live_weight,
'overflow_mode': self.overflow_mode, 'overflow_mode': self.overflow_mode,
'plugin_order': self.plugin_order, 'plugin_order': self.plugin_order,
'excluded_plugins': list(self.excluded_plugins), 'excluded_plugins': list(self.excluded_plugins),
@@ -371,6 +406,15 @@ class VegasModeConfig:
if 'enabled' in vegas_config: if 'enabled' in vegas_config:
self.enabled = vegas_config['enabled'] self.enabled = vegas_config['enabled']
if 'live_in_ticker' in vegas_config:
self.live_in_ticker = bool(vegas_config['live_in_ticker'])
# Clamped exactly as from_config does: a weight below 1 would drop the
# plugin from the rotation, and a huge one starves everything else.
if 'live_weight' in vegas_config:
self.live_weight = max(1, min(10, int(vegas_config['live_weight'])))
if 'favorite_live_weight' in vegas_config:
self.favorite_live_weight = max(
1, min(10, int(vegas_config['favorite_live_weight'])))
if 'scroll_speed' in vegas_config: if 'scroll_speed' in vegas_config:
self.scroll_speed = float(vegas_config['scroll_speed']) self.scroll_speed = float(vegas_config['scroll_speed'])
if 'separator_width' in vegas_config: if 'separator_width' in vegas_config:
+39 -2
View File
@@ -12,6 +12,7 @@ Supports three display modes per plugin:
""" """
import logging import logging
import math
import time import time
import threading import threading
from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING
@@ -30,6 +31,21 @@ if TYPE_CHECKING:
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _percentile(ordered: List[float], fraction: float) -> float:
"""Nearest-rank percentile of an already-sorted list.
Index ceil(n * fraction) - 1, so 100 samples at 0.99 give the 99th-ranked
value. The obvious int(n * fraction) is off by one and, at exactly 100
samples, lands on the maximum -- which is the number already reported
alongside this one as the worst frame, so the two columns would agree
precisely when the sample was smallest.
"""
if not ordered:
return 0.0
index = math.ceil(len(ordered) * fraction) - 1
return ordered[min(len(ordered) - 1, max(0, index))]
class VegasModeCoordinator: class VegasModeCoordinator:
""" """
Orchestrates Vegas scroll mode operation. Orchestrates Vegas scroll mode operation.
@@ -382,6 +398,12 @@ class VegasModeCoordinator:
fps_log_interval = 5.0 # Log FPS every 5 seconds fps_log_interval = 5.0 # Log FPS every 5 seconds
last_fps_log_time = start_time last_fps_log_time = start_time
fps_frame_count = 0 fps_frame_count = 0
# A mean hides stutter completely. At 120fps a five-second window is
# ~600 frames, so a 200ms freeze -- plainly visible on a marquee --
# moves the average from 120.0 to 115.4 and reads as healthy. What a
# viewer actually notices is the worst frame, so track that too.
frame_worst = 0.0
frame_times: List[float] = []
logger.info("Starting Vegas iteration for %.1fs", duration) logger.info("Starting Vegas iteration for %.1fs", duration)
@@ -417,6 +439,11 @@ class VegasModeCoordinator:
frame_elapsed = time.time() - frame_started frame_elapsed = time.time() - frame_started
time.sleep(max(0.0, frame_interval - frame_elapsed)) time.sleep(max(0.0, frame_interval - frame_elapsed))
# Measured before the sleep: time spent working, not pacing.
if frame_elapsed > frame_worst:
frame_worst = frame_elapsed
frame_times.append(frame_elapsed)
# Increment frame count and check for interrupt periodically # Increment frame count and check for interrupt periodically
frame_count += 1 frame_count += 1
fps_frame_count += 1 fps_frame_count += 1
@@ -425,12 +452,16 @@ class VegasModeCoordinator:
current_time = time.time() current_time = time.time()
if current_time - last_fps_log_time >= fps_log_interval: if current_time - last_fps_log_time >= fps_log_interval:
fps = fps_frame_count / (current_time - last_fps_log_time) fps = fps_frame_count / (current_time - last_fps_log_time)
p99 = _percentile(sorted(frame_times), 0.99)
logger.info( logger.info(
"Vegas FPS: %.1f (target: %d, frames: %d)", "Vegas FPS: %.1f (target: %d, frames: %d) p99 %.1fms worst %.1fms",
fps, self.vegas_config.target_fps, fps_frame_count fps, self.vegas_config.target_fps, fps_frame_count,
p99 * 1000.0, frame_worst * 1000.0
) )
last_fps_log_time = current_time last_fps_log_time = current_time
fps_frame_count = 0 fps_frame_count = 0
frame_worst = 0.0
frame_times.clear()
if (self._interrupt_check and if (self._interrupt_check and
frame_count % self._interrupt_check_interval == 0): frame_count % self._interrupt_check_interval == 0):
@@ -497,6 +528,12 @@ class VegasModeCoordinator:
if not self._live_priority_check: if not self._live_priority_check:
return False return False
if self.vegas_config.live_in_ticker:
# The ticker keeps live content rather than yielding to it; the
# extra turns are arranged in the rotation itself, so there is
# nothing to pause for.
return False
try: try:
live_mode = self._live_priority_check() live_mode = self._live_priority_check()
if live_mode: if live_mode:
+139
View File
@@ -406,6 +406,8 @@ class StreamManager:
) )
logger.info("Ordered plugins: %s", ordered_plugins) logger.info("Ordered plugins: %s", ordered_plugins)
ordered_plugins = self._apply_priority_weights(ordered_plugins)
# Atomically update shared state under lock to avoid races with prefetchers # Atomically update shared state under lock to avoid races with prefetchers
with self._buffer_lock: with self._buffer_lock:
self._ordered_plugins = ordered_plugins self._ordered_plugins = ordered_plugins
@@ -417,6 +419,143 @@ class StreamManager:
logger.info("=" * 60) logger.info("=" * 60)
def _plugin_weight(self, plugin_id: str) -> int:
"""Slots per cycle for one plugin.
A plugin may answer for itself via get_vegas_priority_weight() -- the
only way favorite-team awareness can reach here, since the core can see
that a game is live but not whose. When it declines (returns None, the
default), live content earns ``live_weight`` and everything else 1.
"""
plugin = None
try:
plugin = self.plugin_manager.plugins.get(plugin_id)
except (AttributeError, TypeError):
return 1
if plugin is None:
return 1
try:
if hasattr(plugin, 'get_vegas_priority_weight'):
declared = plugin.get_vegas_priority_weight()
if declared is not None:
return max(1, min(10, int(declared)))
except Exception:
# Deliberately falls through to the core's own live check rather
# than demoting to 1. The plugin's weight calculation is broken,
# but has_live_priority() and has_live_content() are separate
# methods guarded separately below -- a plugin that genuinely has
# a live game should still get live_weight for it.
logger.exception("[%s] get_vegas_priority_weight() failed", plugin_id)
try:
if (hasattr(plugin, 'has_live_priority')
and hasattr(plugin, 'has_live_content')
and plugin.has_live_priority()
and plugin.has_live_content()):
return self.config.live_weight
except Exception:
logger.exception("[%s] live-content check failed", plugin_id)
return 1
def _apply_priority_weights(self, ordered: List[str]) -> List[str]:
"""Expand the rotation so weighted plugins take several turns per cycle.
Smooth Weighted Round-Robin, the same scheduler the sports plugins use
to rotate their own games: a plugin of weight N appears N times per
cycle, and the repeats are spaced through the cycle rather than
clumped, so a live score is never three-in-a-row followed by a long
silence.
Returns the input unchanged when nothing is weighted, which is both the
common case and the pre-existing behaviour.
"""
if not ordered or not self.config.live_in_ticker:
return ordered
weights = {pid: self._plugin_weight(pid) for pid in ordered}
total = sum(weights.values())
if total <= len(ordered):
return ordered # nothing boosted; plain round robin
current = {pid: 0 for pid in ordered}
schedule: List[str] = []
for _ in range(total):
for pid in ordered:
current[pid] += weights[pid]
picked = max(current, key=lambda p: current[p])
current[picked] -= total
schedule.append(picked)
schedule = self._unclump_seam(schedule)
boosted = {p: w for p, w in weights.items() if w > 1}
logger.info(
"Vegas rotation weighted: %d slots for %d plugins (boosted: %s)",
len(schedule), len(ordered), boosted)
return schedule
@staticmethod
def _unclump_seam(schedule: List[str]) -> List[str]:
"""Stop the heaviest plugin sitting on both ends of the cycle.
Smooth Weighted Round-Robin spaces repeats well *within* a pass, but
it schedules the heaviest item first and often last too. The strip
loops, so those two are neighbours: the one place the marquee shows
the same plugin twice running is the seam between cycles.
Rotating the list cannot fix this. Rotation preserves the cyclic order
exactly, so it only moves where the seam is drawn, not the adjacency
itself. The trailing entry has to be swapped with one from the middle
whose neighbours differ from it, which breaks the pair without
creating another.
Left alone when no such position exists -- a rotation short enough or
lopsided enough to have none is one where the plugin is unavoidably
adjacent to itself anyway.
"""
if len(schedule) < 3 or schedule[0] != schedule[-1]:
return schedule
repeated = schedule[-1]
size = len(schedule)
def cyclic_doubles(seq) -> int:
return sum(1 for i in range(size) if seq[i] == seq[(i + 1) % size])
def clearance(seq, value) -> int:
"""Smallest cyclic gap between appearances of `value`."""
at = [i for i, v in enumerate(seq) if v == value]
if len(at) < 2:
return size
return min(min((b - a) % size, (a - b) % size)
for i, a in enumerate(at) for b in at[i + 1:])
# Try each swap and judge the result, rather than reasoning about which
# neighbours the two moved elements will end up with. That reasoning is
# where the first version went wrong: it guarded the slot `repeated`
# moves into but not the one the displaced element lands in, so
# ['a','b','c','d','x','y','x','a'] came back ending ['x','x'] -- the
# seam duplicate traded for a fresh one.
best = None
best_clearance = -1
for j in range(1, size - 1):
candidate = list(schedule)
candidate[j], candidate[-1] = candidate[-1], candidate[j]
if cyclic_doubles(candidate):
continue
# Among the repairs that work, prefer the one that leaves the
# boosted plugin most evenly spread; taking the first that merely
# fits moved a repeat from a gap of 7 into a gap of 2.
spread = clearance(candidate, repeated)
if spread > best_clearance:
best, best_clearance = candidate, spread
# None exists when the value is unavoidably adjacent to itself -- a
# plugin holding most of the slots has to be. Schedule it as it is
# rather than refuse.
return best if best is not None else schedule
def _prefetch_content(self, count: int = 1) -> None: def _prefetch_content(self, count: int = 1) -> None:
""" """
Prefetch content for upcoming plugins. Prefetch content for upcoming plugins.
+92
View File
@@ -4,6 +4,7 @@ Centralized error handling for web interface.
Provides helpers for consistent error responses across API endpoints. Provides helpers for consistent error responses across API endpoints.
""" """
import re
from typing import Any, Optional from typing import Any, Optional
from flask import jsonify from flask import jsonify
@@ -16,6 +17,97 @@ from src.logging_config import get_logger
logger = get_logger(__name__) logger = get_logger(__name__)
# Credentials that turn up inside exception text. A requests error quotes the
# URL it failed on, and plugins that authenticate by query string put their key
# there, so echoing an exception verbatim can hand out an API key. Redact the
# value, keep the parameter name -- knowing *which* credential was involved is
# part of the diagnosis.
_REDACT_CREDENTIAL = re.compile(
r'((?:api[_-]?key|access[_-]?token|auth|apikey|key|passwd|password|pwd|'
r'secret|sig|signature|token)["\']?\s*[=:]\s*["\']?)([^\s&"\'<>,}]+)',
re.IGNORECASE,
)
# `Authorization: <scheme> <credential>`. The scheme name is kept because it
# says which kind of credential failed; the credential goes. Any scheme
# matches, not a fixed list: ApiKey, Negotiate, NTLM, AWS4-HMAC-SHA256 and
# whatever a plugin's API invents next are all credentials, and a list would
# silently leak the ones nobody thought of. Not covered by the generic pattern
# above, whose value part stops at whitespace and so would keep the credential
# once a space follows the scheme.
_REDACT_AUTH_HEADER = re.compile(
r'((?:proxy-)?authorization["\']?\s*[=:]\s*["\']?\s*'
r'(?:[A-Za-z][\w.+-]*[ \t]+)?)' # optional scheme name, kept
r'([^\s,"\'<>}]+)', # the credential, redacted
re.IGNORECASE,
)
# Credentials embedded in a URL: https://user:password@host. requests quotes
# the full URL in its exceptions, so this is a realistic leak. The username is
# kept -- it identifies which account failed without being the secret.
_REDACT_URL_USERINFO = re.compile(r'([a-z][a-z0-9+.-]*://[^/\s:@]+:)([^/\s@]+)(@)',
re.IGNORECASE)
# Long enough for an errno string with a path, short enough not to dump a
# parser's worth of context into a JSON field.
_MAX_DETAIL_LENGTH = 400
def describe_exception(exc: BaseException,
max_length: int = _MAX_DETAIL_LENGTH) -> str:
"""
One-line, safe-to-return description of an exception.
The generic "an error occurred; see logs for details" tells a user nothing
and, when the failure is bad enough, the logs are unreachable too: a device
whose storage was failing returned that message from every endpoint
*including* the log viewer, because journalctl could not be executed. The
underlying `[Errno 5] Input/output error` named the fault immediately.
Returns "TypeName: message", credentials redacted and length capped. The
type alone is worth carrying -- a bare PermissionError says more than any
generic sentence.
Args:
exc: The exception to describe
max_length: Truncate beyond this many characters
Returns:
A single-line description, never empty
"""
message = str(exc).strip()
text = f"{type(exc).__name__}: {message}" if message else type(exc).__name__
return redact_text(text, max_length)
def redact_text(text: str, max_length: int = _MAX_DETAIL_LENGTH) -> str:
"""Make arbitrary text safe to hand back over HTTP.
Split out of describe_exception because exceptions are not the only thing
worth returning: a subprocess's stderr, or a message a helper script
printed, is just as useful to a user and just as capable of carrying a
token or a password in it.
Args:
text: The text to redact
max_length: Truncate beyond this many characters
Returns:
A single line, credentials replaced, length capped.
"""
text = text or ''
# Order matters: the URL and header forms are more specific than the
# generic key=value pattern, which would otherwise chew the scheme.
text = _REDACT_URL_USERINFO.sub(r'\1<redacted>\3', text)
text = _REDACT_AUTH_HEADER.sub(r'\1<redacted>', text)
text = _REDACT_CREDENTIAL.sub(r'\1<redacted>', text)
# Collapse newlines/tabs so the detail stays one line in a JSON field.
text = ' '.join(text.split())
if len(text) > max_length:
text = text[:max_length - 1].rstrip() + ''
return text
def create_error_response( def create_error_response(
error_code: ErrorCode, error_code: ErrorCode,
message: str, message: str,
+10 -4
View File
@@ -8,7 +8,9 @@ is_odds_available's ML-blind truth table, the fixed format_odds_summary
gate (money-line-only odds now format), get_odds_for_games, and gate (money-line-only odds now format), get_odds_for_games, and
configuration loading. configuration loading.
No real network: src.base_odds_manager.requests.get is always patched. No real network: requests.Session.get is always patched. The odds path sends
its requests through a session so it can identify itself to ESPN, so patching
the module-level requests.get would no longer intercept anything.
""" """
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
@@ -59,7 +61,7 @@ def manager(cache_manager):
@pytest.fixture @pytest.fixture
def mock_get(): def mock_get():
with patch('src.base_odds_manager.requests.get') as m: with patch('src.base_odds_manager.requests.Session.get') as m:
m.return_value = _make_response({'items': [dict(FULL_ITEM)]}) m.return_value = _make_response({'items': [dict(FULL_ITEM)]})
yield m yield m
@@ -87,7 +89,11 @@ class TestGetOdds:
assert '/events/401/competitions/401/odds' in url assert '/events/401/competitions/401/odds' in url
assert url == ('https://sports.core.api.espn.com/v2/sports/football/' assert url == ('https://sports.core.api.espn.com/v2/sports/football/'
'leagues/nfl/events/401/competitions/401/odds') 'leagues/nfl/events/401/competitions/401/odds')
assert mock_get.call_args.kwargs['timeout'] == 30 # The number matters less than the property: a single stalled request
# must not be able to consume the plugin executor's 30s operation
# budget, since odds are fetched per live game inside update().
assert mock_get.call_args.kwargs['timeout'] == 5
assert mock_get.call_args.kwargs['timeout'] < 30
def test_ncaa_fb_maps_to_college_football(self, manager, mock_get): def test_ncaa_fb_maps_to_college_football(self, manager, mock_get):
manager.get_odds('football', 'ncaa_fb', '401') manager.get_odds('football', 'ncaa_fb', '401')
@@ -355,5 +361,5 @@ class TestLoadConfiguration:
manager = BaseOddsManager(cache_manager, config_manager=config_manager) manager = BaseOddsManager(cache_manager, config_manager=config_manager)
assert manager.update_interval == 3600 assert manager.update_interval == 3600
assert manager.request_timeout == 30 assert manager.request_timeout == 5
assert manager.cache_ttl == 1800 assert manager.cache_ttl == 1800
+146
View File
@@ -0,0 +1,146 @@
"""Tests that one cache directory gets one cleanup thread per process.
The sweep lists a directory and deletes from it, so a second thread over the
same directory only duplicates the scan. Nothing enforced that: every
CacheManager started its own, and since the loop closes over `self`, a
discarded manager could never be collected -- its thread stayed alive and
re-scanned the same directory every 24 hours for the life of the process.
On the dev rig a display process carried three, for one cache directory:
14:22:59.954 display_controller (the real one)
14:22:59.973 startup validation, run 1 (discarded)
14:23:01.055 startup validation, run 2 (discarded)
Startup validation runs twice and built a throwaway manager each time, purely
to read a directory path.
"""
import threading
import pytest
from src.cache_manager import CacheManager
@pytest.fixture(autouse=True)
def _clean_registry():
CacheManager._cleanup_owners.clear()
yield
for owner in list(CacheManager._cleanup_owners.values()):
owner.stop_cleanup_thread()
CacheManager._cleanup_owners.clear()
def _live_cleanup_threads():
return [t for t in threading.enumerate()
if t.name == 'DiskCacheCleanup' and t.is_alive()]
@pytest.fixture
def manager(tmp_path, monkeypatch):
"""A CacheManager pinned to a temp dir, so tests never touch the real one."""
monkeypatch.setattr(CacheManager, '_get_writable_cache_dir',
lambda self: str(tmp_path))
return CacheManager
class TestOneThreadPerDirectory:
def test_a_single_manager_starts_one(self, manager):
before = len(_live_cleanup_threads())
m = manager()
try:
assert len(_live_cleanup_threads()) == before + 1
finally:
m.stop_cleanup_thread()
def test_three_managers_still_start_one(self, manager):
# Exactly the rig's shape: the real manager plus two throwaways.
before = len(_live_cleanup_threads())
managers = [manager() for _ in range(3)]
try:
assert len(_live_cleanup_threads()) == before + 1
finally:
for m in managers:
m.stop_cleanup_thread()
def test_the_first_one_owns_it(self, manager):
first, second = manager(), manager()
try:
assert CacheManager._cleanup_owners[first.cache_dir] is first
assert second._cleanup_thread is None
finally:
first.stop_cleanup_thread()
second.stop_cleanup_thread()
def test_the_survivor_can_take_over(self, manager):
first = manager()
first.stop_cleanup_thread()
assert not _live_cleanup_threads()
second = manager()
try:
# Ownership was released, so the directory is swept again rather
# than being left permanently unclaimed by a dead owner.
assert len(_live_cleanup_threads()) == 1
assert CacheManager._cleanup_owners[second.cache_dir] is second
finally:
second.stop_cleanup_thread()
def test_stopping_a_non_owner_does_not_unclaim_the_directory(self, manager):
first, second = manager(), manager()
try:
second.stop_cleanup_thread() # never owned it
assert CacheManager._cleanup_owners[first.cache_dir] is first
assert len(_live_cleanup_threads()) == 1
finally:
first.stop_cleanup_thread()
def test_separate_directories_get_separate_threads(self, tmp_path, monkeypatch):
a, b = tmp_path / 'a', tmp_path / 'b'
a.mkdir()
b.mkdir()
dirs = iter([str(a), str(b)])
monkeypatch.setattr(CacheManager, '_get_writable_cache_dir',
lambda self: next(dirs))
first, second = CacheManager(), CacheManager()
try:
assert first.cache_dir != second.cache_dir
assert len(_live_cleanup_threads()) == 2
finally:
first.stop_cleanup_thread()
second.stop_cleanup_thread()
def test_no_thread_leaks_across_many_constructions(self, manager):
before = len(_live_cleanup_threads())
made = [manager() for _ in range(12)]
try:
assert len(_live_cleanup_threads()) == before + 1
finally:
for m in made:
m.stop_cleanup_thread()
assert len(_live_cleanup_threads()) == before
class TestValidatorDoesNotBuildItsOwn:
def test_it_uses_the_cache_manager_it_is_given(self, manager):
from src.startup_validator import StartupValidator
shared = manager()
try:
before = len(_live_cleanup_threads())
v = StartupValidator(config_manager=object(), cache_manager=shared)
v._validate_cache_directory()
assert len(_live_cleanup_threads()) == before, (
"validation started another cleanup thread")
finally:
shared.stop_cleanup_thread()
def test_without_one_it_cleans_up_after_itself(self, manager):
from src.startup_validator import StartupValidator
before = len(_live_cleanup_threads())
v = StartupValidator(config_manager=object())
v._validate_cache_directory()
assert len(_live_cleanup_threads()) == before, (
"the fallback manager left its cleanup thread running")
+188
View File
@@ -0,0 +1,188 @@
"""Tests that abandoned cache temp files get collected.
DiskCache.set() writes through tempfile.mkstemp and os.replace, removing its
own temp file in a finally. That covers a failed write, but not a process that
dies between the two -- a SIGKILL, a lost restart race, a power cut, all
ordinary on a Pi. Nothing collected what was left behind: the temp names are
".<key>.json.<random>", and the expiry sweep only listed names ending in
.json, so they accumulated for as long as the card had been in service.
Measured on a live rig before this fix: 76 orphans totalling 1,050 MB -- 81%
of the entire cache directory -- the oldest six months old.
The predicate that decides what to delete is tested harder than the sweep
itself, because a false positive here destroys real data.
"""
import os
import time
import pytest
from src.cache.disk_cache import DiskCache, _ORPHAN_TEMP_MAX_AGE_SECONDS
class FakeStrategy:
@staticmethod
def get_data_type_from_key(key):
return 'default'
POLICIES = {'default': 30}
@pytest.fixture
def cache(tmp_path):
return DiskCache(str(tmp_path))
def _age(path, seconds):
old = time.time() - seconds
os.utime(path, (old, old))
def _write(tmp_path, name, body='{}'):
p = tmp_path / name
p.write_text(body, encoding='utf-8')
return p
class TestWhatCountsAsAnOrphan:
@pytest.mark.parametrize('name', [
'.weather.json.a1b2c3d4',
'.odds_espn_football_nfl_401.json.xyz00000',
'.a.json.b',
])
def test_our_temp_files_are_orphans(self, name):
assert DiskCache._is_orphaned_temp(name)
@pytest.mark.parametrize('name', [
'weather.json', # real data
'.weather.json', # a dotted key that completed
'.gitignore', # not ours
'.hidden', # not ours
'weather.json.bak', # no leading dot: someone else's
'.json.abc', # no key between the dot and .json.
'.weather.json.', # no random component
'notes.txt',
])
def test_everything_else_is_left_alone(self, name):
assert not DiskCache._is_orphaned_temp(name)
def test_the_names_set_actually_creates_are_matched(self, cache, tmp_path):
"""Guard against the predicate and the writer drifting apart."""
created = []
real = os.replace
def capture(src, dst):
created.append(os.path.basename(src))
return real(src, dst)
import src.cache.disk_cache as mod
mod.os.replace = capture
try:
cache.set('weather', {'v': 1})
finally:
mod.os.replace = real
assert created, "set() did not go through the temp-file path"
assert all(DiskCache._is_orphaned_temp(n) for n in created), created
class TestTheSweep:
def test_an_old_orphan_is_removed(self, cache, tmp_path):
p = _write(tmp_path, '.weather.json.a1b2c3d4', 'x' * 5000)
_age(p, _ORPHAN_TEMP_MAX_AGE_SECONDS + 60)
stats = cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert not p.exists()
assert stats['orphan_temp_files_deleted'] == 1
assert stats['space_freed_bytes'] >= 5000
def test_an_in_flight_write_is_not_snatched_away(self, cache, tmp_path):
# The whole risk of this sweep: deleting a temp file another thread is
# about to os.replace into place.
p = _write(tmp_path, '.weather.json.inflight')
cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert p.exists()
def test_real_cache_files_survive(self, cache, tmp_path):
fresh = _write(tmp_path, 'weather.json')
dotted = _write(tmp_path, '.weather.json')
_age(dotted, _ORPHAN_TEMP_MAX_AGE_SECONDS + 60)
cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert fresh.exists()
assert dotted.exists(), "a completed .json was treated as a temp file"
def test_unrelated_dotfiles_survive(self, cache, tmp_path):
keep = _write(tmp_path, '.gitignore')
_age(keep, 400 * 86400)
cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert keep.exists()
def test_expiry_still_works_alongside_it(self, cache, tmp_path):
stale = _write(tmp_path, 'old.json')
_age(stale, 40 * 86400) # past the 30-day default
orphan = _write(tmp_path, '.old.json.zz999999')
_age(orphan, _ORPHAN_TEMP_MAX_AGE_SECONDS + 60)
stats = cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert not stale.exists()
assert not orphan.exists()
assert stats['files_deleted'] == 2
assert stats['orphan_temp_files_deleted'] == 1
def test_the_rig_scenario(self, cache, tmp_path):
"""76 orphans of assorted ages, none of them reachable before."""
for i in range(76):
p = _write(tmp_path, '.sched_%d.json.r%06d' % (i, i), 'x' * 1000)
_age(p, (i + 2) * 86400)
keep = _write(tmp_path, 'sched.json')
stats = cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert stats['orphan_temp_files_deleted'] == 76
assert keep.exists()
assert not list(tmp_path.glob('.sched_*'))
# The summary line is "<deleted>/<scanned>", so an orphan that is
# deleted but never counted as scanned renders as "76/1".
assert stats['files_scanned'] == 77
assert stats['files_deleted'] <= stats['files_scanned']
def test_deleted_never_exceeds_scanned(self, cache, tmp_path):
p = _write(tmp_path, '.only.json.a1b2c3d4')
_age(p, _ORPHAN_TEMP_MAX_AGE_SECONDS + 60)
stats = cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert stats['files_deleted'] == 1
assert stats['files_scanned'] == 1
def test_a_missing_file_mid_sweep_is_not_an_error(self, cache, tmp_path):
p = _write(tmp_path, '.weather.json.a1b2c3d4')
_age(p, _ORPHAN_TEMP_MAX_AGE_SECONDS + 60)
import src.cache.disk_cache as mod
real = mod.os.path.getsize
def vanish(path):
if path.endswith('.a1b2c3d4'):
os.remove(path)
raise FileNotFoundError(path)
return real(path)
mod.os.path.getsize = vanish
try:
stats = cache.cleanup_expired_files(FakeStrategy(), POLICIES)
finally:
mod.os.path.getsize = real
assert stats['errors'] == 0
+130
View File
@@ -0,0 +1,130 @@
"""Tests that a per-entry ttl actually controls expiry.
Regression under test: `CacheManager.set(key, data, ttl=...)` stored the value
and no read path ever consulted it. Expiry came from a `max_age` inferred from
substrings in the key ("live", "odds", "stock"), so every caller passing `ttl=`
-- 48 sites across the plugins and 4 in the core -- was writing a number that
did nothing. The old docstring admitted as much: "stored for compatibility but
expiration is still controlled via max_age when reading".
Measured against a real device's cache (8,873 entries carrying a ttl), the
inferred value and the intended one disagreed almost everywhere:
stocks max_age 600 vs ttl 1800 4903 entries
news max_age 3600 vs ttl 600 1770 entries
odds max_age 1800 vs ttl 3600 1301 entries
images max_age 300 vs ttl 2592000 20 entries
No `sports_live` entry carries a ttl, so live scores keep their inferred
30-second freshness either way.
"""
import time
import pytest
from src.cache.memory_cache import MemoryCache
from src.cache.disk_cache import DiskCache
@pytest.fixture
def disk(tmp_path):
return DiskCache(cache_dir=str(tmp_path))
def _record(ttl=None, age=0.0):
rec = {"data": {"v": 1}, "timestamp": time.time() - age}
if ttl is not None:
rec["ttl"] = ttl
return rec
class TestDiskCacheHonoursTtl:
def test_ttl_longer_than_max_age_keeps_the_entry(self, disk):
# The odds case: written wanting an hour, expired at 30 minutes.
disk.set("odds_espn_football_nfl_401", _record(ttl=3600, age=1900))
assert disk.get("odds_espn_football_nfl_401", max_age=1800) is not None
def test_ttl_shorter_than_max_age_expires_the_entry(self, disk):
# The news case: written wanting 10 minutes, kept for an hour.
disk.set("news_NHL_1", _record(ttl=600, age=900))
assert disk.get("news_NHL_1", max_age=3600) is None
def test_without_a_ttl_max_age_still_applies(self, disk):
disk.set("plain_key", _record(age=400))
assert disk.get("plain_key", max_age=300) is None
disk.set("plain_key2", _record(age=100))
assert disk.get("plain_key2", max_age=300) is not None
def test_a_fresh_entry_within_its_ttl_survives(self, disk):
disk.set("k", _record(ttl=600, age=10))
assert disk.get("k", max_age=30) is not None
def test_ttl_zero_expires_immediately(self, disk):
# 0 means zero seconds, not "forever" -- max_age=None is how a caller
# asks for no expiry.
disk.set("k", _record(ttl=0, age=1))
assert disk.get("k", max_age=99999) is None
@pytest.mark.parametrize("bad", ["600", None, True, False, -5, {"a": 1}])
def test_a_nonsense_ttl_falls_back_to_max_age(self, disk, bad):
# Including bools: True is an int in Python and must not become a 1s ttl.
rec = _record(age=400)
rec["ttl"] = bad
disk.set("k_%s" % type(bad).__name__, rec)
assert disk.get("k_%s" % type(bad).__name__, max_age=300) is None
class TestMemoryCacheHonoursTtl:
def test_ttl_longer_than_max_age_keeps_the_entry(self):
m = MemoryCache()
m.set("k", _record(ttl=3600))
m._timestamps["k"] = time.time() - 1900
assert m.get("k", max_age=1800) is not None
def test_ttl_shorter_than_max_age_expires_the_entry(self):
m = MemoryCache()
m.set("k", _record(ttl=600))
m._timestamps["k"] = time.time() - 900
assert m.get("k", max_age=3600) is None
def test_without_a_ttl_max_age_still_applies(self):
m = MemoryCache()
m.set("k", _record())
m._timestamps["k"] = time.time() - 400
assert m.get("k", max_age=300) is None
def test_both_layers_agree(self, tmp_path):
"""A record must not be live in one layer and expired in the other."""
rec = _record(ttl=3600, age=1900)
d = DiskCache(cache_dir=str(tmp_path))
d.set("k", rec)
m = MemoryCache()
m.set("k", rec)
m._timestamps["k"] = rec["timestamp"]
assert (d.get("k", max_age=1800) is not None) == (m.get("k", max_age=1800) is not None)
class TestEndToEnd:
def test_set_then_get_respects_the_ttl(self, tmp_path, monkeypatch):
"""The behaviour a caller of CacheManager.set(ttl=...) expects."""
from src.cache_manager import CacheManager
cm = CacheManager()
cm._disk_cache_component = DiskCache(cache_dir=str(tmp_path))
cm._memory_cache_component = MemoryCache()
cm.set("odds_espn_football_nfl_401", {"spread": 6.5}, ttl=3600)
# Age the stored record past the inferred max_age for odds (1800s) but
# within the ttl the caller asked for.
path = cm._disk_cache_component.get_cache_path("odds_espn_football_nfl_401")
import json
rec = json.load(open(path))
rec["timestamp"] = time.time() - 1900
json.dump(rec, open(path, "w"))
cm._memory_cache_component.clear() if hasattr(
cm._memory_cache_component, "clear") else None
got = cm.get_with_auto_strategy("odds_espn_football_nfl_401")
assert got is not None, "the ttl the caller asked for was ignored"
+42
View File
@@ -237,3 +237,45 @@ class TestDisplayManagerDoubleSided:
suppress_test_pattern=True) suppress_test_pattern=True)
assert dm.set_brightness(70) is True assert dm.set_brightness(70) is True
assert mock_rgb_matrix['matrix_instance'].brightness == 70 assert mock_rgb_matrix['matrix_instance'].brightness == 70
class TestDisplayManagerOrientation:
"""The orientation setting composes onto pixel_mapper_config for panels
mounted upside down, without disturbing a custom pixel_mapper_config."""
def _config(self, **hardware_overrides):
config = {
'display': {
'hardware': {
'rows': 32, 'cols': 64, 'chain_length': 2, 'parallel': 1,
'hardware_mapping': 'adafruit-hat-pwm', 'brightness': 90,
},
'runtime': {'gpio_slowdown': 2},
},
'timezone': 'UTC',
'plugin_system': {'plugins_directory': 'plugins'},
}
config['display']['hardware'].update(hardware_overrides)
return config
def test_default_orientation_leaves_pixel_mapper_config_untouched(self, mock_rgb_matrix):
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'false'}):
DisplayManager(self._config(), suppress_test_pattern=True)
options = mock_rgb_matrix['options_class'].return_value
assert options.pixel_mapper_config == ''
def test_orientation_180_appends_rotate_mapper(self, mock_rgb_matrix):
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'false'}):
DisplayManager(self._config(orientation='180'), suppress_test_pattern=True)
options = mock_rgb_matrix['options_class'].return_value
assert options.pixel_mapper_config == 'Rotate:180'
def test_orientation_180_composes_with_existing_pixel_mapper_config(self, mock_rgb_matrix):
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'false'}):
DisplayManager(self._config(orientation='180', pixel_mapper_config='U-mapper'),
suppress_test_pattern=True)
options = mock_rgb_matrix['options_class'].return_value
assert options.pixel_mapper_config == 'U-mapper;Rotate:180'
+224
View File
@@ -0,0 +1,224 @@
"""Tests that startup does not wait indefinitely for plugins to fetch data.
DisplayController.__init__ calls _update_modules() once, to populate plugin
data before the first frame. It walks every loaded plugin in turn, and each
update blocks the calling thread for up to the executor's 30s timeout, so the
uncapped total is the sum of every slow plugin on the system.
Profiled on a live rig with py-spy, the main thread sat 9.34s in
display_controller._update_modules
-> plugin_executor.execute_update
-> execute_with_timeout -> threading.join
and the controller's own log put the full pass at 82 seconds on the worst
boot measured (55 and 26 on the two before). The panel shows nothing for all
of it.
Nothing is lost by stopping early: a plugin that has never updated is
immediately due, so run_scheduled_updates() collects it seconds later with the
display already running.
"""
import os
import time
from unittest.mock import Mock
import pytest
# display_controller imports display_manager, which binds the hardware
# rgbmatrix module unless EMULATOR=true is set before import (same convention
# as test_display_controller_vegas_tick.py).
os.environ.setdefault("EMULATOR", "true")
from src.display_controller import ( # noqa: E402
DisplayController, _INITIAL_UPDATE_BUDGET_SECONDS,
_MIN_INITIAL_UPDATE_TIMEOUT_SECONDS)
class FakeExecutor:
"""Records which plugins were updated, and can make some of them slow."""
def __init__(self, cost=0.0, slow=()):
self.updated = []
self.cost = cost
self.slow = set(slow)
def execute_update(self, plugin, plugin_id, timeout=None):
self.updated.append(plugin_id)
if plugin_id in self.slow:
time.sleep(self.cost)
return True
@pytest.fixture
def tiny_floor(monkeypatch):
"""Shrink the "worth starting" floor so timing tests stay quick."""
import src.display_controller as mod
monkeypatch.setattr(mod, "_MIN_INITIAL_UPDATE_TIMEOUT_SECONDS", 0.01)
def _controller(plugin_ids, executor):
c = DisplayController.__new__(DisplayController)
c.plugin_manager = Mock()
# Both attributes, because _update_modules reads
# `loaded_plugins or plugins` and an empty dict is falsy.
c.plugin_manager.loaded_plugins = {pid: Mock() for pid in plugin_ids}
c.plugin_manager.plugins = dict(c.plugin_manager.loaded_plugins)
c.plugin_manager.plugin_executor = executor
c.plugin_manager.plugin_last_update = {}
c.plugin_manager.health_tracker = None
return c
class TestTheBudgetIsRespected:
def test_without_a_deadline_every_plugin_is_updated(self):
ex = FakeExecutor()
_controller(['a', 'b', 'c'], ex)._update_modules()
assert ex.updated == ['a', 'b', 'c']
def test_a_passed_deadline_stops_the_pass(self):
ex = FakeExecutor()
_controller(['a', 'b', 'c'], ex)._update_modules(deadline=time.time() - 1)
assert ex.updated == [], "updated %r after the deadline" % ex.updated
def test_slow_plugins_do_not_drag_in_the_rest(self, tiny_floor):
# One plugin burns the whole budget; the remainder must be left alone
# rather than each adding its own wait.
ex = FakeExecutor(cost=0.3, slow={'slow'})
c = _controller(['slow'] + ['p%d' % i for i in range(20)], ex)
started = time.time()
c._update_modules(deadline=started + 0.2)
elapsed = time.time() - started
assert ex.updated == ['slow'], "updated %r" % ex.updated
# Bounded by the one in-flight update, not by twenty more.
assert elapsed < 1.0, "%.2fs" % elapsed
def test_a_generous_deadline_still_gets_everything(self):
ex = FakeExecutor()
c = _controller(['a', 'b', 'c'], ex)
c._update_modules(deadline=time.time() + 30)
assert ex.updated == ['a', 'b', 'c']
def test_the_deadline_is_checked_before_each_plugin(self, tiny_floor):
# Not just once up front: the budget can be spent partway through.
ex = FakeExecutor(cost=0.15, slow={'a', 'b', 'c', 'd'})
c = _controller(['a', 'b', 'c', 'd'], ex)
c._update_modules(deadline=time.time() + 0.2)
assert 0 < len(ex.updated) < 4, "updated %r" % ex.updated
class TestThePassIsBoundedInPractice:
def test_the_last_plugin_cannot_overrun_the_budget(self):
# Checking the deadline before each plugin is not enough on its own:
# one that starts with a moment left could still block for the
# executor's full timeout. On the rig that turned a 20s budget into a
# 31.8s pass, so the remaining budget is passed down as the timeout.
seen = []
class Executor:
def execute_update(self, plugin, plugin_id, timeout=None):
seen.append(timeout)
return True
c = _controller(['a', 'b', 'c'], Executor())
deadline = time.time() + 5
c._update_modules(deadline=deadline)
assert seen and all(t is not None for t in seen), seen
assert all(t <= 5.01 for t in seen), seen
# The exact remainder, never clamped up: clamping would let the pass
# run past its deadline. Anything below the floor is deferred instead,
# so what does start always has a usable slot.
assert all(t >= _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS for t in seen), seen
def test_without_a_deadline_the_executor_default_is_left_alone(self):
seen = []
class Executor:
def execute_update(self, plugin, plugin_id, timeout=None):
seen.append(timeout)
return True
_controller(['a'], Executor())._update_modules()
assert seen == [None], seen
class TestTheBudgetItself:
def test_it_is_short_enough_to_be_worth_having(self):
# The measured uncapped worst case was 82s; a budget near that would
# not bound anything.
assert _INITIAL_UPDATE_BUDGET_SECONDS <= 30
def test_it_is_long_enough_for_a_quick_plugin_or_two(self):
assert _INITIAL_UPDATE_BUDGET_SECONDS >= 5
class TestNothingIsSilentlyDropped:
def test_deferred_plugins_are_named_in_the_log(self, caplog):
ex = FakeExecutor()
c = _controller(['a', 'b'], ex)
with caplog.at_level('INFO'):
c._update_modules(deadline=time.time() - 1)
text = "\n".join(r.getMessage() for r in caplog.records)
assert 'a' in text and 'b' in text, text
assert 'budget' in text.lower(), text
def test_nothing_is_logged_when_all_of_them_ran(self, caplog):
ex = FakeExecutor()
c = _controller(['a'], ex)
with caplog.at_level('INFO'):
c._update_modules(deadline=time.time() + 30)
assert not any('budget' in r.getMessage().lower() for r in caplog.records)
class TestItDoesNotBreakTheOrdinaryPaths:
def test_no_plugin_manager_is_harmless(self):
c = DisplayController.__new__(DisplayController)
c.plugin_manager = None
c._update_modules(deadline=time.time() - 1) # must not raise
def test_an_empty_plugin_set_is_harmless(self):
ex = FakeExecutor()
_controller([], ex)._update_modules(deadline=time.time() + 5)
assert ex.updated == []
class TestTooLittleBudgetDefersRatherThanClamps:
def test_a_plugin_starting_below_the_floor_is_deferred(self):
ex = FakeExecutor()
c = _controller(['a'], ex)
# Just under the floor: previously this was clamped up to the floor and
# run anyway, which pushed the pass past its deadline.
c._update_modules(
deadline=time.time() + _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS - 0.05)
assert ex.updated == [], "started a plugin it could not give a slot to"
def test_a_plugin_starting_above_the_floor_still_runs(self):
ex = FakeExecutor()
c = _controller(['a'], ex)
c._update_modules(
deadline=time.time() + _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS + 1)
assert ex.updated == ['a']
def test_the_timeout_is_the_remainder_not_the_floor(self):
seen = []
class Executor:
def execute_update(self, plugin, plugin_id, timeout=None):
seen.append(timeout)
return True
c = _controller(['a'], Executor())
c._update_modules(deadline=time.time() + 9)
assert seen and 8.5 <= seen[0] <= 9.01, seen
def test_the_pass_cannot_outlast_its_deadline(self, tiny_floor):
# Every plugin sleeps well past the budget; the deferral keeps the
# whole pass inside it rather than overrunning by a floor's worth.
ex = FakeExecutor(cost=0.4, slow={'a', 'b', 'c', 'd', 'e'})
c = _controller(['a', 'b', 'c', 'd', 'e'], ex)
started = time.time()
c._update_modules(deadline=started + 0.5)
assert time.time() - started < 1.2, "%.2fs" % (time.time() - started)
+190
View File
@@ -0,0 +1,190 @@
"""Tests the startup screen that shows while plugins fetch their first data.
That screen is on the panel for the whole initial-update window, and on a
headless Pi it is the only place the device's address appears without going
looking for it -- so it now carries the address as well as "Initializing".
Two things have to hold. It must fit every supported panel: the old fixed
8px PressStart2P drew "Initializing" 96px wide at x=10, which ran off the
side of a 64px panel before an address was ever added. And the lookup must be
cheap, because this runs on the startup path that the rest of this change
exists to shorten.
"""
import os
import time
from PIL import Image, ImageDraw, ImageFont
import pytest
os.environ.setdefault("EMULATOR", "true")
from src.display_manager import DisplayManager # noqa: E402
SIZES = [(64, 32), (128, 32), (128, 64), (256, 32), (512, 64)]
class FakeMatrix:
def __init__(self, width, height):
self.width, self.height = width, height
def _manager(width, height):
dm = DisplayManager.__new__(DisplayManager)
dm.image = Image.new('RGB', (width, height))
dm.draw = ImageDraw.Draw(dm.image)
dm.matrix = FakeMatrix(width, height)
dm.font = ImageFont.truetype('assets/fonts/PressStart2P-Regular.ttf', 8)
return dm
def _layout(dm, lines):
"""The geometry _draw_startup_banner uses."""
font = dm._fitting_font(lines, dm.matrix.width - 2)
line_height = dm.draw.textbbox((0, 0), "Ag", font=font)[3] + 1
top = max(1, (dm.matrix.height - line_height * len(lines)) // 2)
widths = [dm.draw.textlength(t, font=font) for t in lines]
return font, widths, top, top + line_height * len(lines)
def _render_over_pattern(width, height, lines):
"""Draw the test pattern, then the banner over it, as startup does."""
dm = _manager(width, height)
dm.draw.rectangle([0, 0, width - 1, height - 1], outline=(255, 0, 0))
dm.draw.line([0, 0, width - 1, height - 1], fill=(0, 255, 0))
dm._draw_startup_banner(lines, width, height)
return dm
class TestTheAddressLookup:
def test_it_never_reports_loopback(self):
# A loopback address on the panel would be actively misleading -- it is
# not something anyone can browse to.
ip = DisplayManager._local_ip()
assert ip is None or not ip.startswith("127."), ip
def test_it_looks_like_an_address_when_there_is_one(self):
ip = DisplayManager._local_ip()
if ip is None:
pytest.skip("host has no routable address")
parts = ip.split(".")
assert len(parts) == 4 and all(p.isdigit() for p in parts), ip
def test_it_is_cheap_enough_for_the_startup_path(self):
DisplayManager._local_ip() # warm anything cacheable
started = time.perf_counter()
for _ in range(20):
DisplayManager._local_ip()
per_call = (time.perf_counter() - started) / 20
# `hostname -I` with its 2s timeout, which the web launcher uses, would
# be thousands of times this.
assert per_call < 0.05, "%.1f ms per call" % (per_call * 1000)
def test_it_returns_none_rather_than_raising(self, monkeypatch):
import src.display_manager as mod
def no_network(*a, **k):
raise OSError("network is unreachable")
monkeypatch.setattr(mod.socket, "socket", no_network)
assert DisplayManager._local_ip() is None
class TestItFitsEveryPanel:
@pytest.mark.parametrize("width,height", SIZES)
def test_both_lines_fit_with_an_address(self, width, height):
dm = _manager(width, height)
_font, widths, top, bottom = _layout(dm, ["Initializing", "255.255.255.255"])
assert all(w <= width - 2 for w in widths), (width, widths)
assert bottom <= height and top >= 0, (top, bottom, height)
@pytest.mark.parametrize("width,height", SIZES)
def test_it_still_fits_with_no_address(self, width, height):
dm = _manager(width, height)
_font, widths, _top, bottom = _layout(dm, ["Initializing"])
assert all(w <= width - 2 for w in widths), (width, widths)
assert bottom <= height, (bottom, height)
def test_the_smallest_panel_drops_to_a_narrower_font(self):
# The regression this guards: PressStart2P at 8px is 96px wide for
# "Initializing", which does not fit 64px however it is positioned.
dm = _manager(64, 32)
font, widths, _t, _b = _layout(dm, ["Initializing", "10.0.20.104"])
assert font is not dm.font, "kept a font that cannot fit"
assert max(widths) <= 62, widths
def test_a_roomy_panel_keeps_the_larger_font(self):
dm = _manager(256, 32)
font, _w, _t, _b = _layout(dm, ["Initializing", "10.0.20.104"])
assert font is dm.font, "needlessly shrank on a panel with room"
class TestPlacement:
@pytest.mark.parametrize("width,height", SIZES)
def test_the_lines_are_centred(self, width, height):
dm = _manager(width, height)
lines = ["Initializing", "10.0.20.104"]
_font, widths, _t, _b = _layout(dm, lines)
for w in widths:
left = max(0, (width - w) // 2)
assert abs((left + (left + w)) - width) <= 2, (left, w, width)
def test_the_address_sits_under_the_word(self):
dm = _manager(128, 64)
font, _w, top, bottom = _layout(dm, ["Initializing", "10.0.20.104"])
line_height = dm.draw.textbbox((0, 0), "Ag", font=font)[3] + 1
assert bottom - top == line_height * 2
class TestItIsActuallyReadable:
"""The point of the address is that someone can read it off the wall."""
@pytest.mark.parametrize("width,height", SIZES)
def test_the_diagonal_does_not_cross_the_text(self, width, height):
lines = ["Initializing", "10.0.20.104"]
dm = _render_over_pattern(width, height, lines)
_font, widths, top, bottom = _layout(dm, lines)
# textlength returns a float, so these must be floored before they
# can index pixels.
block_width = int(max(widths))
left = int(max(0, (width - block_width) // 2))
px = dm.image.load()
green = 0
for y in range(int(top), min(int(bottom), height)):
for x in range(left, min(left + block_width, width)):
r, g, b = px[x, y]
if g > 128 and r < 128 and b < 128:
green += 1
assert green == 0, "%d green pixels behind the text at %dx%d" % (
green, width, height)
@pytest.mark.parametrize("width,height", SIZES)
def test_the_text_stays_pure_blue(self, width, height):
# Not a style choice. The pattern lights one channel per element --
# red border, green diagonal, blue text -- so a glance says whether
# led_rgb_sequence is right: wire it BGR and the border comes up blue
# and this text red. White text would light all three and destroy the
# only blue reference on the screen.
dm = _render_over_pattern(width, height, ["Initializing", "10.0.20.104"])
px = dm.image.load()
blue = sum(1 for y in range(height) for x in range(width)
if px[x, y] == (0, 0, 255))
assert blue > 20, "only %d blue pixels at %dx%d" % (blue, width, height)
white = sum(1 for y in range(height) for x in range(width)
if px[x, y] == (255, 255, 255))
assert white == 0, "%d white pixels would muddy the channel check" % white
def test_each_element_lights_one_channel(self):
# The whole point of the pattern: three pure primaries on screen.
dm = _render_over_pattern(128, 64, ["Initializing", "10.0.20.104"])
seen = set(dm.image.getdata())
assert (255, 0, 0) in seen, "no pure red border"
assert (0, 255, 0) in seen, "no pure green diagonal"
assert (0, 0, 255) in seen, "no pure blue text"
def test_nothing_is_drawn_for_no_lines(self):
dm = _manager(128, 64)
before = dm.image.tobytes()
dm._draw_startup_banner([], 128, 64)
assert dm.image.tobytes() == before
+162
View File
@@ -0,0 +1,162 @@
"""Tests that a slow ESPN cannot take a whole plugin update with it.
Odds are fetched per live game from inside SportsLive.update(), with show_odds
defaulting on, and the plugin executor kills an operation at 30s. The odds
request timeout was also 30s, so one stalled request consumed the entire budget
and the update carrying every game's score was killed:
00:43:43 ERROR plugin football-scoreboard operation timed out after 30.0s
01:43:43 ERROR plugin football-scoreboard operation timed out after 30.0s
Invisible out of season -- preseason week 1 returns a single game -- and a
Sunday slate is around sixteen.
The request now goes through a session that identifies the caller, so the
tests patch `manager.session.get` rather than the module's `requests.get`.
"""
from unittest.mock import Mock
import requests
from src.base_odds_manager import BaseOddsManager
PLUGIN_BUDGET = 30.0 # PluginExecutor(default_timeout=30.0)
def _manager(cache=None):
cache = cache or Mock()
cache.get_with_auto_strategy.return_value = None
return BaseOddsManager(cache_manager=cache, config_manager=None)
def _timing_out(manager):
"""Point the manager's session at a request that always times out."""
manager.session.get = Mock(side_effect=requests.exceptions.Timeout("x"))
return manager.session.get
def _returning(manager, payload):
resp = Mock()
resp.json.return_value = payload
resp.raise_for_status.return_value = None
manager.session.get = Mock(return_value=resp)
return manager.session.get
class TestRequestTimeout:
def test_leaves_room_in_the_operation_budget(self):
assert _manager().request_timeout < PLUGIN_BUDGET / 2
def test_the_timeout_is_the_one_actually_used(self):
m = _manager()
get = _timing_out(m)
m.get_odds("football", "nfl", "401")
assert get.call_args.kwargs["timeout"] == m.request_timeout
class TestIdentifiesItselfToEspn:
"""ESPN 403s python-requests' default agent, and bare custom tokens.
What it accepts is a token carrying a URL that says who is calling. This
path used a bare requests.get and so sent the default -- the one thing
known to be rejected. Everything else in the tree that talks to ESPN
already sends the header below.
"""
def test_the_user_agent_names_the_project_and_links_to_it(self):
ua = _manager().session.headers["User-Agent"]
assert "python-requests" not in ua
assert "LEDMatrix" in ua
assert "github.com/ChuckBuilds/LEDMatrix" in ua
def test_it_is_the_same_agent_the_rest_of_the_tree_sends(self):
# Compared against the live value rather than a copied literal, so the
# two cannot drift apart the next time ESPN moves the goalposts.
from src.common.api_helper import APIHelper
assert (_manager().session.headers["User-Agent"]
== APIHelper().session.headers["User-Agent"])
def test_the_header_reaches_the_request(self):
m = _manager()
get = _returning(m, {})
m._extract_espn_data = Mock(return_value=None)
m.get_odds("football", "nfl", "401")
# Sent via the session, so it applies without being passed per-call.
assert get.call_count == 1
assert "User-Agent" in m.session.headers
def test_no_retry_adapter_multiplies_the_timeout(self):
# api_helper mounts a retrying adapter; this path must not, or a 5s
# timeout becomes 15s and the budget fix is undone.
m = _manager()
for adapter in m.session.adapters.values():
retries = getattr(adapter, "max_retries", None)
assert getattr(retries, "total", 0) in (0, None), (
"odds session mounts a retrying adapter (total=%r); retries "
"multiply request_timeout" % getattr(retries, "total", None))
class TestSlowEspnCannotKillTheUpdate:
def test_one_failure_stops_the_rest_of_the_slate_hitting_the_network(self):
m = _manager()
get = _timing_out(m)
for i in range(16): # a full slate, one game at a time
m.get_odds("football", "nfl", "4018730%02d" % i)
assert get.call_count == 1, (
"%d games each paid the timeout; the breaker should have stopped "
"after the first" % get.call_count)
def test_worst_case_slate_stays_inside_the_budget(self):
m = _manager()
assert m.request_timeout * 1 < PLUGIN_BUDGET
def test_recovery_is_automatic(self):
m = _manager()
import src.base_odds_manager as mod
real_monotonic = mod.time.monotonic
clock = {"t": 1000.0}
try:
mod.time.monotonic = lambda: clock["t"]
get = _timing_out(m)
m.get_odds("football", "nfl", "401")
assert m._skip_network_until > clock["t"], "breaker did not open"
clock["t"] += 1
before = get.call_count
m.get_odds("football", "nfl", "402")
assert get.call_count == before, "should not have retried"
clock["t"] += m._FAILURE_COOLDOWN
m.get_odds("football", "nfl", "403")
assert get.call_count > before, "never retried"
finally:
mod.time.monotonic = real_monotonic
def test_a_healthy_fetch_clears_the_breaker(self):
m = _manager()
m._skip_network_until = 0.0
m._extract_espn_data = Mock(return_value=None)
_returning(m, {})
m.get_odds("football", "nfl", "401")
assert m._skip_network_until == 0.0
def test_a_403_opens_the_breaker_rather_than_hammering(self):
# raise_for_status raises HTTPError, a RequestException -- so a wrong
# or missing agent backs off instead of 403ing once per game.
m = _manager()
resp = Mock()
resp.raise_for_status.side_effect = requests.exceptions.HTTPError("403")
m.session.get = Mock(return_value=resp)
m.get_odds("football", "nfl", "401")
assert m._skip_network_until > 0.0
def test_the_stale_cache_fallback_still_works(self):
# The failing request must still hand back whatever was cached; only
# the *subsequent* games skip the network.
cache = Mock()
cache.get_with_auto_strategy.side_effect = [None, {"details": "stale"}]
m = BaseOddsManager(cache_manager=cache, config_manager=None)
_timing_out(m)
assert m.get_odds("football", "nfl", "401") == {"details": "stale"}
+241
View File
@@ -0,0 +1,241 @@
"""
Getting Started checklist: what the server decides, and what it must not.
The timezone step used to tick server-side when the saved timezone differed
from the shipped default, OR-ed with the saved city. That made the step
unsatisfiable for anyone genuinely in the default zone (the card nagged
forever), and let a saved city tick it off while the timezone was still wrong.
The step is now verified in the browser against its own zone, so the server's
only job is to hand over the configured value and stay out of the decision.
These tests pin that contract: the panel-size step still reflects config, the
timezone step never pre-ticks, it carries the configured zone, and the city
has no influence on it.
"""
import copy
import re
import sys
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from flask import Flask
PROJECT_ROOT = Path(__file__).parent.parent
sys.path.insert(0, str(PROJECT_ROOT))
BASE_CONFIG = {
"timezone": "America/New_York",
"location": {"city": "Tampa", "state": "Florida", "country": "US"},
"display": {
"hardware": {"rows": 32, "cols": 64, "chain_length": 2, "parallel": 1},
"runtime": {},
"double_sided": {"enabled": False},
"vegas_scroll": {"plugin_order": [], "excluded_plugins": []},
"plugin_rotation_order": [],
},
"plugin_system": {},
"schedule": {},
"dim_schedule": {},
"sync": {},
}
def render(config):
"""Render the overview partial against one config, as app.py would."""
base = PROJECT_ROOT / "web_interface"
app = Flask(
__name__,
template_folder=str(base / "templates"),
static_folder=str(base / "static"),
)
app.config["TESTING"] = True
from web_interface.blueprints import pages_v3 as pv
# pages_v3 is a module-level singleton shared across the test process;
# restore whatever the previous test left on it.
original_cm = getattr(pv.pages_v3, "config_manager", None)
original_pm = getattr(pv.pages_v3, "plugin_manager", None)
mock_cm = MagicMock()
mock_cm.load_config.return_value = config
mock_cm.get_raw_file_content.return_value = config
pv.pages_v3.config_manager = mock_cm
mock_pm = MagicMock()
mock_pm.plugins = {}
mock_pm.get_all_plugin_info.return_value = []
mock_pm.get_plugin_display_modes.side_effect = lambda pid: []
pv.pages_v3.plugin_manager = mock_pm
app.register_blueprint(pv.pages_v3, url_prefix="")
try:
resp = app.test_client().get("/partials/overview")
assert resp.status_code == 200, resp.status_code
return resp.get_data(as_text=True)
finally:
pv.pages_v3.config_manager = original_cm
pv.pages_v3.plugin_manager = original_pm
def timezone_step(body):
"""The checklist <button> for the timezone step."""
match = re.search(r"<button[^>]*data-check=\"timezone\"[^>]*>", body)
assert match, "timezone step not found in the rendered checklist"
return match.group(0)
def config_with(**overrides):
config = copy.deepcopy(BASE_CONFIG)
for key, value in overrides.items():
config[key] = value
return config
@pytest.mark.parametrize(
"timezone",
["America/New_York", "America/Los_Angeles", "Europe/Madrid", "Asia/Kolkata"],
)
def test_timezone_step_never_pre_ticks_server_side(timezone):
"""The browser owns this decision; the server must not pre-empt it.
The default zone is in the list deliberately: that is the case the old
default-comparison could never tick.
"""
step = timezone_step(render(config_with(timezone=timezone)))
assert 'data-done="0"' in step, step
@pytest.mark.parametrize(
"timezone",
["America/New_York", "Europe/Madrid", "Pacific/Auckland"],
)
def test_timezone_step_carries_the_configured_zone(timezone):
"""JS compares data-tz against the browser, so it has to be the real value."""
assert f'data-tz="{timezone}"' in timezone_step(render(config_with(timezone=timezone)))
def test_city_does_not_influence_the_timezone_step():
"""The coupling this change removes: city said nothing about the timezone,
and OR-ing it let a saved city tick the step off with the zone still wrong.
timezone_step() returns the opening tag only, so this compares the state
the step is in -- data-done and data-tz -- and not the label, which does
still show the configured city as context and so differs between the two.
"""
tampa = timezone_step(render(config_with(
location={"city": "Tampa", "state": "Florida", "country": "US"})))
seattle = timezone_step(render(config_with(
location={"city": "Seattle", "state": "Washington", "country": "US"})))
assert tampa == seattle
def test_missing_timezone_leaves_the_step_open():
"""Nothing saved means nothing to verify: the step stays unticked and the
JS bails on the empty value rather than comparing against ''."""
step = timezone_step(render(config_with(timezone="")))
assert 'data-tz=""' in step
assert 'data-done="0"' in step
def test_zone_comparison_asks_for_the_time_of_day():
"""Guard on the Intl options, which look like a stylistic choice.
dateStyle/timeStyle are late additions (Firefox shipped them in 91). An
implementation that does not know them ignores them and formats the date
alone -- which compares New York, Chicago and Madrid as equal and ticks
the step for a timezone that is plainly wrong. Explicit numeric fields
have been in Intl since ECMA-402 v1.
"""
template = (PROJECT_ROOT / "web_interface" / "templates" / "v3"
/ "partials" / "overview.html").read_text()
body = template[template.index("function sameZone"):]
body = body[:body.index("}())")]
# The comment above the options names dateStyle/timeStyle to explain why
# they are not used, so match on code only.
body = "\n".join(line for line in body.splitlines()
if not line.lstrip().startswith("//"))
assert "dateStyle" not in body and "timeStyle" not in body, (
"zone comparison must not depend on dateStyle/timeStyle")
for field in ("hour:", "minute:", "year:", "month:", "day:"):
assert field in body, f"zone comparison dropped {field!r}"
def test_zone_comparison_samples_both_sides_of_dst():
"""One instant is not enough, and the shortfall is invisible for months.
America/New_York and America/Lima hold the same offset all winter, so a
check against now alone ticks the step in January for a panel that runs an
hour off from March. The comparison has to sample instants either side of
DST -- mid-January and mid-July, which covers both hemispheres.
"""
template = (PROJECT_ROOT / "web_interface" / "templates" / "v3"
/ "partials" / "overview.html").read_text()
body = template[template.index("function sameZone"):]
body = body[:body.index("}())")]
code = "\n".join(line for line in body.splitlines()
if not line.lstrip().startswith("//"))
assert "Date.UTC" in code, (
"zone comparison samples only the current instant, so zones that "
"coincide seasonally would read as equal")
assert code.count("Date.UTC") >= 2, "expected an instant either side of DST"
def _stamp(zone, instant):
"""The JS comparison's algorithm, for pinning what it must decide.
There is no JS runtime here (and the repo has no JS test infra), so this
mirrors sameZone rather than executing it: same instants, same wall-clock
equality. It records the verdicts the shipped code has to reach.
"""
from zoneinfo import ZoneInfo
return instant.astimezone(ZoneInfo(zone)).strftime("%m/%d/%Y %H:%M")
@pytest.mark.parametrize(
"left,right,equivalent",
[
# Aliases: one zone under two names.
("Asia/Calcutta", "Asia/Kolkata", True),
("Europe/Kiev", "Europe/Kyiv", True),
# Same rules year-round: either renders the same times, so a panel set
# to one and browsed from the other is correctly configured.
("America/New_York", "America/Toronto", True),
# Coincide in winter only -- the case a single-instant check gets wrong.
("America/New_York", "America/Lima", False),
("America/Phoenix", "America/Los_Angeles", False),
("Australia/Sydney", "Pacific/Guadalcanal", False),
# Plainly different.
("America/New_York", "America/Chicago", False),
("America/New_York", "Europe/Madrid", False),
],
)
def test_which_zone_pairs_must_count_as_the_same(left, right, equivalent):
from datetime import datetime
from zoneinfo import ZoneInfo
year = 2026
instants = [datetime(year, 1, 15, 12, tzinfo=ZoneInfo("UTC")),
datetime(year, 7, 15, 12, tzinfo=ZoneInfo("UTC"))]
matched = all(_stamp(left, at) == _stamp(right, at) for at in instants)
assert matched is equivalent, (
f"{left} vs {right}: sampling both seasons gave {matched}")
@pytest.mark.parametrize(
"hardware,expected",
[
({"rows": 32, "cols": 64, "chain_length": 2, "parallel": 1}, "1"),
({"rows": 0, "cols": 0, "chain_length": 0, "parallel": 1}, "0"),
],
)
def test_panel_size_step_still_reflects_config(hardware, expected):
"""Regression guard: the hardware step is still decided server-side."""
config = config_with()
config["display"]["hardware"] = hardware
body = render(config)
match = re.search(r"<button[^>]*data-tab=\"display\"[^>]*>", body)
assert match, "panel-size step not found"
assert f'data-done="{expected}"' in match.group(0), match.group(0)
+64
View File
@@ -0,0 +1,64 @@
"""Tests the percentile used by the Vegas frame-time log line.
The FPS line reports p99 next to the worst frame, and the point of having both
is that they say different things: p99 is the bad-but-ordinary frame, worst is
the outlier. The obvious index, int(n * 0.99), is off by one and at exactly
100 samples selects the maximum -- so the two columns would report the same
number precisely when the sample was smallest.
"""
import math
import pytest
from src.vegas_mode.coordinator import _percentile
class TestNearestRank:
def test_a_hundred_samples_do_not_return_the_maximum(self):
ordered = [float(i) for i in range(100)] # 0..99
assert _percentile(ordered, 0.99) == 98.0
assert _percentile(ordered, 0.99) != max(ordered)
def test_it_matches_the_nearest_rank_definition(self):
for n in (1, 2, 3, 10, 99, 100, 101, 600, 1000):
ordered = [float(i) for i in range(n)]
expected = ordered[min(n - 1, max(0, math.ceil(n * 0.99) - 1))]
assert _percentile(ordered, 0.99) == expected, n
@pytest.mark.parametrize('fraction,expected', [
(0.0, 0.0), # first
(0.5, 49.0), # median, nearest-rank
(1.0, 99.0), # last
])
def test_other_fractions(self, fraction, expected):
assert _percentile([float(i) for i in range(100)], fraction) == expected
class TestEdges:
def test_empty_is_zero_not_an_error(self):
# The loop calls this before any frame has been timed.
assert _percentile([], 0.99) == 0.0
def test_a_single_sample_is_itself(self):
assert _percentile([4.2], 0.99) == 4.2
def test_it_never_indexes_past_the_end(self):
for n in range(1, 50):
_percentile([float(i) for i in range(n)], 1.0) # must not raise
class TestItSaysSomethingUsefulAboutFrames:
def test_one_freeze_does_not_drag_p99_up(self):
# 599 healthy frames and one 3.2s freeze: p99 should still describe
# the healthy population, while the worst frame is reported separately.
frames = [0.0083] * 599 + [3.2]
p99 = _percentile(sorted(frames), 0.99)
assert p99 == pytest.approx(0.0083), p99
assert max(frames) == 3.2
def test_sustained_slowness_does_move_it(self):
# Ten percent of frames slow is not an outlier, it is the shape of the
# distribution, and p99 must reflect that.
frames = [0.0083] * 540 + [0.05] * 60
assert _percentile(sorted(frames), 0.99) == pytest.approx(0.05)
+300
View File
@@ -0,0 +1,300 @@
"""Tests that live content can take extra turns inside the Vegas ticker.
Vegas was a strict round robin -- every plugin exactly once per cycle -- and
live content did not appear in it at all, because the display controller
refused to run the ticker while anything was live. With a dozen plugins
enabled that left a live score either absent or minutes stale.
Two things change, both off by default. `live_in_ticker` keeps the marquee
running instead of yielding to a full-screen takeover, and the rotation is
expanded by Smooth Weighted Round-Robin so a weighted plugin gets several
slots per cycle, spaced through it rather than clumped.
Weights are per plugin, not per game: a scoreboard showing four live games
still occupies one slot at a time and rotates its own games within it.
"""
from unittest.mock import Mock
import pytest
from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.stream_manager import StreamManager
class FakePlugin:
"""A plugin that can fail in each place independently.
hook_raises and live_raises are separate because they mean different
things: a broken weight calculation should still leave the core's own
live-content check usable, while a plugin that cannot answer whether it is
live at all has nothing left to fall back on.
"""
def __init__(self, live=False, declared=None, raises=False,
hook_raises=False, live_raises=False):
self._live = live
self._declared = declared
self._hook_raises = hook_raises or raises
self._live_raises = live_raises or raises
self.enabled = True
def has_live_priority(self):
if self._live_raises:
raise RuntimeError("cannot say whether I am live")
return self._live
def has_live_content(self):
return self._live
def get_vegas_priority_weight(self):
if self._hook_raises:
raise RuntimeError("weight calculation blew up")
return self._declared
def _manager(plugins, **cfg):
config = VegasModeConfig(live_in_ticker=cfg.pop('live_in_ticker', True), **cfg)
pm = Mock()
pm.plugins = plugins
sm = StreamManager.__new__(StreamManager)
sm.config = config
sm.plugin_manager = pm
return sm
def _counts(schedule):
return {p: schedule.count(p) for p in set(schedule)}
def _max_gap(schedule, plugin_id):
"""Largest gap between consecutive appearances, wrapping around."""
at = [i for i, p in enumerate(schedule) if p == plugin_id]
if len(at) < 2:
return len(schedule)
gaps = [b - a for a, b in zip(at, at[1:])]
gaps.append(len(schedule) - at[-1] + at[0])
return max(gaps)
class TestWeightsComeFromTheRightPlace:
def test_a_quiet_plugin_gets_one_slot(self):
sm = _manager({'clock': FakePlugin()})
assert sm._plugin_weight('clock') == 1
def test_live_content_earns_the_configured_weight(self):
sm = _manager({'mlb': FakePlugin(live=True)}, live_weight=4)
assert sm._plugin_weight('mlb') == 4
def test_a_plugin_may_answer_for_itself(self):
# The only route for favorite-team awareness: the core can see that a
# game is live, not whose.
sm = _manager({'mlb': FakePlugin(live=True, declared=7)}, live_weight=3)
assert sm._plugin_weight('mlb') == 7
def test_declaring_none_defers_to_the_core(self):
sm = _manager({'mlb': FakePlugin(live=True, declared=None)}, live_weight=3)
assert sm._plugin_weight('mlb') == 3
def test_a_declared_weight_is_clamped(self):
sm = _manager({'a': FakePlugin(declared=99), 'b': FakePlugin(declared=0)})
assert sm._plugin_weight('a') == 10
assert sm._plugin_weight('b') == 1
def test_a_plugin_that_raises_everywhere_weighs_one(self):
sm = _manager({'bad': FakePlugin(raises=True)})
assert sm._plugin_weight('bad') == 1
def test_a_broken_hook_still_earns_the_live_boost(self):
# The hook is only how a plugin asks for *more* than live_weight.
# Losing it should cost the favorite distinction, not the live boost:
# has_live_priority/has_live_content are separate and still work.
sm = _manager({'mlb': FakePlugin(live=True, hook_raises=True)},
live_weight=4)
assert sm._plugin_weight('mlb') == 4
def test_a_broken_hook_on_a_quiet_plugin_weighs_one(self):
sm = _manager({'clock': FakePlugin(live=False, hook_raises=True)},
live_weight=4)
assert sm._plugin_weight('clock') == 1
def test_a_plugin_that_cannot_say_whether_it_is_live_weighs_one(self):
# Nothing left to fall back on, so no boost.
sm = _manager({'mlb': FakePlugin(live=True, live_raises=True)},
live_weight=4)
assert sm._plugin_weight('mlb') == 1
def test_an_unknown_plugin_weighs_one(self):
assert _manager({})._plugin_weight('ghost') == 1
class TestTheSchedule:
def test_nothing_weighted_leaves_the_order_untouched(self):
order = ['weather', 'clock', 'news']
sm = _manager({p: FakePlugin() for p in order})
assert sm._apply_priority_weights(order) == order
def test_off_by_default_the_order_is_untouched(self):
order = ['weather', 'mlb', 'news']
sm = _manager({'weather': FakePlugin(), 'mlb': FakePlugin(live=True),
'news': FakePlugin()}, live_in_ticker=False, live_weight=3)
assert sm._apply_priority_weights(order) == order
def test_a_live_plugin_takes_its_share_of_slots(self):
order = ['weather', 'mlb', 'news', 'clock']
sm = _manager({'weather': FakePlugin(), 'mlb': FakePlugin(live=True),
'news': FakePlugin(), 'clock': FakePlugin()},
live_weight=3)
schedule = sm._apply_priority_weights(order)
counts = _counts(schedule)
assert counts['mlb'] == 3, counts
assert counts['weather'] == counts['news'] == counts['clock'] == 1, counts
assert len(schedule) == 6
def test_every_plugin_still_appears(self):
# A boost must not starve anything out of the cycle.
order = ['a', 'b', 'c', 'd', 'e', 'f']
plugins = {p: FakePlugin() for p in order}
plugins['a'] = FakePlugin(live=True, declared=10)
sm = _manager(plugins)
schedule = sm._apply_priority_weights(order)
assert set(schedule) == set(order), set(order) - set(schedule)
def test_nothing_doubles_across_the_cycle_seam(self):
# The strip loops, so the last slot neighbours the first. Smooth
# Weighted Round-Robin schedules the heaviest item first and often
# last too, which put the one clump the algorithm exists to avoid at
# the one place a within-cycle check cannot see.
order = ['baseball', 'weather', 'geochron', 'flights', 'stocks',
'oftheday', 'youtube', 'stocknews', 'leaderboard',
'countdown', 'odds', 'f1', 'football', 'music']
plugins = {p: FakePlugin() for p in order}
plugins['baseball'] = FakePlugin(live=True, declared=5)
plugins['football'] = FakePlugin(live=True, declared=3)
schedule = _manager(plugins)._apply_priority_weights(order)
n = len(schedule)
doubles = [schedule[i] for i in range(n)
if schedule[i] == schedule[(i + 1) % n]]
assert not doubles, "%r repeats across the seam in %r" % (doubles, schedule)
def test_the_seam_repair_keeps_every_slot(self):
order = ['a', 'b', 'c', 'd', 'e', 'f']
plugins = {p: FakePlugin() for p in order}
plugins['a'] = FakePlugin(live=True, declared=4)
schedule = _manager(plugins)._apply_priority_weights(order)
assert _counts(schedule)['a'] == 4, _counts(schedule)
assert sorted(schedule) == sorted(
['a'] * 4 + ['b', 'c', 'd', 'e', 'f']), schedule
def test_the_repair_uses_the_widest_gap(self):
# Moving the trailing repeat into the first slot that merely fits
# undoes the spacing: on a 28-slot rotation that turned a gap of 7
# into a gap of 2, which is more clumped than the seam ever was.
order = ['a'] + ['p%d' % i for i in range(13)]
plugins = {p: FakePlugin() for p in order}
plugins['a'] = FakePlugin(live=True, declared=4)
schedule = _manager(plugins)._apply_priority_weights(order)
at = [i for i, p in enumerate(schedule) if p == 'a']
gaps = [b - a for a, b in zip(at, at[1:])]
gaps.append(len(schedule) - at[-1] + at[0])
ideal = len(schedule) / len(at)
assert min(gaps) >= ideal / 2, "gaps %r for ideal %.1f" % (gaps, ideal)
def test_an_unavoidable_double_is_left_alone(self):
# Five of seven slots are the same plugin, so it must neighbour
# itself. Better to schedule it than to refuse or loop forever.
order = ['a', 'b', 'c']
plugins = {p: FakePlugin() for p in order}
plugins['a'] = FakePlugin(live=True, declared=5)
schedule = _manager(plugins)._apply_priority_weights(order)
assert _counts(schedule) == {'a': 5, 'b': 1, 'c': 1}, _counts(schedule)
assert set(schedule) == {'a', 'b', 'c'}
def test_the_repair_never_creates_a_new_double(self):
# The first version guarded the slot the repeated value moves *into*
# but not the one the displaced element lands in, so this traded the
# seam duplicate for a fresh one and came back ending ['x', 'x'].
sm = _manager({})
out = sm._unclump_seam(['a', 'b', 'c', 'd', 'x', 'y', 'x', 'a'])
n = len(out)
doubles = [out[i] for i in range(n) if out[i] == out[(i + 1) % n]]
assert not doubles, "%r in %r" % (doubles, out)
assert sorted(out) == sorted(['a', 'b', 'c', 'd', 'x', 'y', 'x', 'a'])
def test_the_last_two_slots_are_a_usable_swap(self):
# Reasoning about indices said this candidate was unsafe because
# schedule[j] is schedule[-2]; after the swap its neighbour is the
# repeated value, not itself. Refusing it left the only repair this
# schedule has on the table.
assert _manager({})._unclump_seam(['a', 'b', 'c', 'a']) == ['a', 'b', 'a', 'c']
def test_no_seam_schedule_is_ever_made_worse(self):
import random
sm = _manager({})
random.seed(11)
checked = 0
for size in range(3, 10):
for _ in range(400):
original = [random.choice('abcd') for _ in range(size)]
if original[0] != original[-1]:
continue
checked += 1
out = sm._unclump_seam(list(original))
n = len(out)
before = sum(1 for i in range(n)
if original[i] == original[(i + 1) % n])
after = sum(1 for i in range(n) if out[i] == out[(i + 1) % n])
assert after <= before, (original, out)
assert sorted(out) == sorted(original), (original, out)
assert checked > 100, "the generator stopped producing seam cases"
def test_a_schedule_too_short_to_repair_is_returned_as_is(self):
sm = _manager({})
assert sm._unclump_seam(['a', 'a']) == ['a', 'a']
assert sm._unclump_seam(['a']) == ['a']
assert sm._unclump_seam([]) == []
def test_a_schedule_with_no_seam_clash_is_untouched(self):
sm = _manager({})
plain = ['a', 'b', 'c', 'a', 'd']
assert sm._unclump_seam(plain) == plain
def test_repeats_are_spread_not_clumped(self):
# The point of Smooth Weighted Round-Robin. Three-in-a-row followed by
# a long silence would be worse than not boosting at all.
order = ['weather', 'mlb', 'news', 'clock', 'stocks', 'f1']
plugins = {p: FakePlugin() for p in order}
plugins['mlb'] = FakePlugin(live=True)
sm = _manager(plugins, live_weight=3)
schedule = sm._apply_priority_weights(order)
assert _counts(schedule)['mlb'] == 3
# Evenly spread over 8 slots means a gap of about 3, never 6.
assert _max_gap(schedule, 'mlb') <= 4, schedule
# And never twice running.
assert not any(a == b == 'mlb' for a, b in zip(schedule, schedule[1:])), schedule
def test_a_favorite_outranks_another_live_game(self):
order = ['weather', 'mlb', 'nhl']
sm = _manager({'weather': FakePlugin(),
'mlb': FakePlugin(live=True, declared=5),
'nhl': FakePlugin(live=True)}, live_weight=2)
counts = _counts(sm._apply_priority_weights(order))
assert counts['mlb'] == 5 and counts['nhl'] == 2 and counts['weather'] == 1, counts
def test_an_empty_rotation_is_harmless(self):
assert _manager({})._apply_priority_weights([]) == []
class TestConfigParsing:
def test_defaults_preserve_todays_behaviour(self):
cfg = VegasModeConfig.from_config({})
assert cfg.live_in_ticker is False
assert cfg.live_weight == 3 and cfg.favorite_live_weight == 5
@pytest.mark.parametrize("given,expected", [(0, 1), (-4, 1), (99, 10), (4, 4)])
def test_weights_are_clamped(self, given, expected):
cfg = VegasModeConfig.from_config(
{'display': {'vegas_scroll': {'live_weight': given}}})
assert cfg.live_weight == expected
+248
View File
@@ -0,0 +1,248 @@
"""Tests for surfacing the underlying error in web responses.
Regression under test: every failing endpoint returned "An error occurred; see
logs for details" and nothing else. On a device whose storage was failing that
sentence came back from the restart action, from /system/status, and from
/logs -- the log viewer itself -- because journalctl could not be executed. The
exception underneath said `[Errno 5] Input/output error: 'systemctl'`, which
names the fault outright, and nine handlers were discarding it entirely rather
than even logging it.
"""
import pytest
from src.web_interface.error_handler import describe_exception
class TestDescribeException:
def test_names_the_type_and_message(self):
detail = describe_exception(OSError(5, "Input/output error", "systemctl"))
assert detail == "OSError: [Errno 5] Input/output error: 'systemctl'"
def test_the_reported_failure_is_legible(self):
# The whole point: this string is the diagnosis.
assert "Input/output error" in describe_exception(
OSError(5, "Input/output error", "systemctl"))
def test_a_bare_exception_still_names_its_type(self):
# A PermissionError with no message still says more than "unknown".
assert describe_exception(PermissionError()) == "PermissionError"
assert describe_exception(Exception()) == "Exception"
def test_message_is_kept_when_present(self):
assert describe_exception(ValueError("bad port")) == "ValueError: bad port"
class TestCredentialRedaction:
"""Exception text quotes URLs, and plugins authenticate by query string."""
@pytest.mark.parametrize("secret_text,leaked", [
("failed: https://api.x.com/v1?api_key=SEC123&city=Tampa", "SEC123"),
("token=abcdef123456 was rejected", "abcdef123456"),
("connect failed password=hunter2", "hunter2"),
("GET /?access_token=zzz999", "zzz999"),
('{"secret": "topsecret"}', "topsecret"),
# requests quotes the URL it failed on, and both of these forms turn
# up in real client exceptions.
("401 for https://user:hunter2@example.com/api", "hunter2"),
("headers: {'Authorization': 'Bearer eyJ.SECRET.sig'}", "eyJ.SECRET.sig"),
("Authorization: Basic dXNlcjpwYXNzd29yZA==", "dXNlcjpwYXNzd29yZA=="),
("Proxy-Authorization: Bearer ptok999", "ptok999"),
# Any scheme, not a fixed list -- a list silently leaks whatever it
# does not name, and plugin APIs invent their own.
("Authorization: ApiKey SECRET123", "SECRET123"),
("Authorization: Negotiate YIIZnegotiateblob", "YIIZnegotiateblob"),
("Authorization: NTLM TlRMTVNTUAAB", "TlRMTVNTUAAB"),
("authorization: barecredential", "barecredential"),
])
def test_credentials_never_reach_the_response(self, secret_text, leaked):
detail = describe_exception(RuntimeError(secret_text))
assert leaked not in detail
assert "<redacted>" in detail
def test_the_parameter_name_survives_redaction(self):
# Knowing *which* credential was involved is part of the diagnosis.
detail = describe_exception(RuntimeError("https://x/y?api_key=SEC123"))
assert "api_key" in detail
def test_unknown_schemes_keep_their_name(self):
for scheme in ("ApiKey", "Negotiate", "NTLM", "AWS4-HMAC-SHA256"):
detail = describe_exception(
RuntimeError("Authorization: %s SECRETVALUE" % scheme))
assert scheme in detail, detail
assert "SECRETVALUE" not in detail, detail
def test_auth_scheme_and_username_survive(self):
# Which kind of credential, and whose, without the credential itself.
assert "Bearer" in describe_exception(
RuntimeError("Authorization: Bearer eyJ.SECRET.sig"))
assert "user" in describe_exception(
RuntimeError("https://user:hunter2@example.com"))
def test_non_secret_context_is_preserved(self):
detail = describe_exception(RuntimeError("https://api.x.com/v1?city=Tampa"))
assert "city=Tampa" in detail
assert "<redacted>" not in detail
class TestBounds:
def test_long_messages_are_truncated(self):
detail = describe_exception(ValueError("x" * 5000))
assert len(detail) <= 400
def test_newlines_are_collapsed_to_one_line(self):
detail = describe_exception(ValueError("line one\nline two\tthree"))
assert "\n" not in detail and "\t" not in detail
assert detail == "ValueError: line one line two three"
def test_custom_length_is_honoured(self):
assert len(describe_exception(ValueError("y" * 500), max_length=50)) <= 50
class TestHandlersCarryDetail:
"""The response shape callers actually see."""
def test_no_api_v3_handler_discards_its_exception(self):
"""Every generic-message handler must log a traceback and return detail.
Nine of them bound `e` and never used it, so the promised log entry was
never written either. Checking merely that *something* was logged is
too weak -- a `logger.info("failed")` would satisfy it while throwing
the exception away just as completely, so this asserts the two things
that actually make the failure diagnosable: an error-level record with
the traceback, and the sanitized detail in the response.
"""
import ast
src = open("web_interface/blueprints/api_v3.py").read()
tree = ast.parse(src)
generic = "An error occurred; see logs for details"
def logs_a_traceback(handler):
"""An error/exception-level log call carrying exc_info."""
for call in [n for n in ast.walk(handler) if isinstance(n, ast.Call)]:
func = call.func
if not isinstance(func, ast.Attribute):
continue
if func.attr == "exception": # implies exc_info
return True
if func.attr not in ("error", "critical"):
continue
if any(kw.arg == "exc_info" and getattr(kw.value, "value", False) is True
for kw in call.keywords):
return True
return False
def describes_this_exception(node, bound):
"""A describe_exception(<bound>) call anywhere under `node`."""
for call in [n for n in ast.walk(node) if isinstance(n, ast.Call)]:
if not (isinstance(call.func, ast.Name)
and call.func.id == "describe_exception"):
continue
if bound is None:
return True # bare `except:` cannot name it; accept
if any(isinstance(a, ast.Name) and a.id == bound
for a in call.args):
return True
return False
def returns_the_detail(handler):
"""The detail must be inside what the handler actually returns.
Looking anywhere in the handler is too weak: a handler could
compute describe_exception(e), drop it on the floor, and return the
generic message with no details field, while still passing. So the
call has to appear within a `return` expression.
"""
returns = [n for n in ast.walk(handler) if isinstance(n, ast.Return)]
if not returns:
return False
return all(describes_this_exception(r, handler.name) for r in returns)
offenders = []
for h in [n for n in ast.walk(tree) if isinstance(n, ast.ExceptHandler)]:
seg = ast.get_source_segment(src, h) or ""
if generic not in seg:
continue
missing = []
if not logs_a_traceback(h):
missing.append("error-level log with exc_info")
if not returns_the_detail(h):
missing.append("describe_exception(e) in the response")
if missing:
offenders.append((h.lineno, missing))
assert not offenders, (
"handlers returning the generic message without %s: %r"
% ("both a traceback log and the detail", offenders))
def test_client_errors_keep_their_own_status(self):
"""A 405 must not be reported as a server-side UNKNOWN_ERROR.
Werkzeug's HTTPExceptions subclass Exception, so the catch-all saw them
too: a GET on a POST-only route came back 500 "an error occurred",
which tells the caller nothing and blames the wrong side. Found while
probing a device whose POST-only config endpoints answered every GET
with UNKNOWN_ERROR.
"""
from flask import Flask, jsonify
from werkzeug.exceptions import HTTPException
app = Flask(__name__)
@app.errorhandler(Exception)
def handle(error):
if isinstance(error, HTTPException):
return jsonify({
"status": "error",
"error_code": (error.name or "HTTP_ERROR").upper().replace(" ", "_"),
"message": error.description,
}), error.code or 500
return jsonify({
"status": "error",
"error_code": "UNKNOWN_ERROR",
"message": "An error occurred; see logs for details",
"details": describe_exception(error),
}), 500
@app.route("/only-post", methods=["POST"])
def only_post():
return jsonify({"ok": True})
@app.route("/boom")
def boom():
raise OSError(5, "Input/output error", "systemctl")
client = app.test_client()
resp = client.get("/only-post")
assert resp.status_code == 405, "a wrong method must stay a 405"
assert resp.get_json()["error_code"] == "METHOD_NOT_ALLOWED"
# A genuine server fault still reports as one, with its detail.
resp = client.get("/boom")
assert resp.status_code == 500
assert "Input/output error" in resp.get_json()["details"]
def test_global_handler_reports_the_underlying_error(self):
from flask import Flask, jsonify
app = Flask(__name__)
@app.errorhandler(Exception)
def handle(error):
return jsonify({
"status": "error",
"error_code": "UNKNOWN_ERROR",
"message": "An error occurred; see logs for details",
"details": describe_exception(error),
}), 500
@app.route("/boom")
def boom():
raise OSError(5, "Input/output error", "systemctl")
client = app.test_client()
body = client.get("/boom").get_json()
assert body["error_code"] == "UNKNOWN_ERROR"
assert "Input/output error" in body["details"]
@@ -0,0 +1,408 @@
"""Tests the calendar plugin's OAuth and calendar-listing endpoints.
The plugin's config UI advertised a three-step setup, but only step 1 existed
on the server. Step 3's picker fetched /api/v3/plugins/calendar/list-calendars,
which was never registered, so Flask fell through to the global 404 handler and
the user saw "Resource not found" with nothing to say which resource. Step 2
had no endpoint either, and no field in the schema at all, even though the
plugin ships calendar_registration.py written expressly for a web-driven
two-step flow.
These cover the two new routes: that they exist, that they fail with something
actionable rather than a bare 404, and that the shapes the widgets consume are
what the server actually sends.
"""
import json
import pickle
import sys
from pathlib import Path
import pytest
project_root = Path(__file__).parent.parent.parent
sys.path.insert(0, str(project_root))
from web_interface.blueprints import api_v3 as mod # noqa: E402
@pytest.fixture
def client(monkeypatch, tmp_path):
"""A test client whose calendar plugin lives in tmp_path."""
from flask import Flask
plugin_dir = tmp_path / 'calendar'
plugin_dir.mkdir()
app = Flask(__name__)
app.register_blueprint(mod.api_v3, url_prefix='/api/v3')
app.config['TESTING'] = True
monkeypatch.setattr(mod, '_calendar_plugin_dir', lambda: plugin_dir)
with app.test_client() as c:
c.plugin_dir = plugin_dir
yield c
@pytest.fixture
def uninstalled(monkeypatch):
from flask import Flask
app = Flask(__name__)
app.register_blueprint(mod.api_v3, url_prefix='/api/v3')
app.config['TESTING'] = True
monkeypatch.setattr(mod, '_calendar_plugin_dir', lambda: None)
with app.test_client() as c:
yield c
class TestTheRoutesExistAtAll:
"""The original bug: the URLs the widgets call were not registered."""
def test_list_calendars_is_routed(self, client):
response = client.get('/api/v3/plugins/calendar/list-calendars')
# Reaching the handler is the whole point; what it then says about
# missing setup is TestItSaysWhatIsWrong's business.
assert response.status_code != 404, "still unrouted"
assert response.get_json()['message'] != 'Resource not found'
def test_authenticate_is_routed(self, client):
response = client.post('/api/v3/plugins/calendar/authenticate', json={})
assert response.status_code != 404, "still unrouted"
assert response.get_json()['message'] != 'Resource not found'
def test_both_urls_match_what_the_widgets_request(self):
# The widgets hardcode these; a rename on either side reintroduces the
# original bug silently.
picker = Path(project_root) / 'web_interface/static/v3/js/widgets/google-calendar-picker.js'
oauth = Path(project_root) / 'web_interface/static/v3/js/widgets/google-oauth.js'
assert '/api/v3/plugins/calendar/list-calendars' in picker.read_text(encoding='utf-8')
assert '/api/v3/plugins/calendar/authenticate' in oauth.read_text(encoding='utf-8')
source = (Path(project_root) / 'web_interface/blueprints/api_v3.py').read_text(encoding='utf-8')
assert "'/plugins/calendar/list-calendars'" in source
assert "'/plugins/calendar/authenticate'" in source
def test_the_oauth_widget_is_dispatched_not_rendered_as_a_text_box(self):
# The string branch of the config template dispatches on an allow-list
# of widget names; anything missing from it silently falls through to a
# plain <input type="text">. That produced two boxes on the calendar
# page -- the widget's own, and a stray one for the same field -- and
# no way to tell which to paste into.
template = (Path(project_root)
/ 'web_interface/templates/v3/partials/plugin_config.html'
).read_text(encoding='utf-8')
allow_list_line = [ln for ln in template.splitlines()
if "str_widget in [" in ln]
assert allow_list_line, "the string widget allow-list moved"
assert "'google-oauth'" in allow_list_line[0], allow_list_line[0]
def test_the_widget_script_is_served(self):
base = (Path(project_root) / 'web_interface/templates/v3/base.html'
).read_text(encoding='utf-8')
assert 'widgets/google-oauth.js' in base
def test_the_status_line_is_announced(self):
# Every message the widget gives arrives after an async call, so a
# screen reader hears nothing unless the element is a live region.
widget = (Path(project_root)
/ 'web_interface/static/v3/js/widgets/google-oauth.js'
).read_text(encoding='utf-8')
# Both attributes must be on the *status* element. Searching for them
# separately would pass with each on a different node, which announces
# nothing.
assert "status.setAttribute('role', 'status')" in widget, widget[:0]
assert "status.setAttribute('aria-live', 'polite')" in widget
def test_the_paste_box_has_an_accessible_name(self):
# A visible label is not enough on its own: without the association the
# input's only name is a placeholder, which vanishes on focus -- which
# is exactly when the value is being pasted.
widget = (Path(project_root)
/ 'web_interface/static/v3/js/widgets/google-oauth.js'
).read_text(encoding='utf-8')
# The binding is what matters, not that both lines exist: a `for` and
# an `id` that disagree leave the input just as anonymous. Both must
# go through the same identifier.
import re as _re
for_target = _re.search(r"codeLabel\.setAttribute\('for',\s*(\w+)\)", widget)
id_source = _re.search(r"codeInput\.id\s*=\s*(\w+)", widget)
assert for_target and id_source, (for_target, id_source)
assert for_target.group(1) == id_source.group(1), (
"label points at %r but the input is %r"
% (for_target.group(1), id_source.group(1)))
def test_the_failed_page_is_called_out_loudly(self):
# The loopback redirect lands on a browser error page at exactly the
# moment the user has to act. In small grey text it gets missed and the
# flow reads as broken while it is working.
widget = (Path(project_root)
/ 'web_interface/static/v3/js/widgets/google-oauth.js'
).read_text(encoding='utf-8')
assert 'expected' in widget.lower()
assert 'amber' in widget, "the warning is not visually distinguished"
class TestItSaysWhatIsWrong:
def test_listing_without_a_token_asks_for_step_2(self, client):
response = client.get('/api/v3/plugins/calendar/list-calendars')
assert response.status_code == 400
body = response.get_json()
assert body['status'] == 'error'
assert 'step 2' in body['message'].lower(), body['message']
def test_authenticating_without_credentials_asks_for_step_1(self, client):
response = client.post('/api/v3/plugins/calendar/authenticate', json={})
assert response.status_code == 400
assert 'step 1' in response.get_json()['message'].lower()
def test_an_uninstalled_plugin_says_so(self, uninstalled):
for response in (
uninstalled.get('/api/v3/plugins/calendar/list-calendars'),
uninstalled.post('/api/v3/plugins/calendar/authenticate', json={}),
):
assert response.status_code == 404
# A 404 here is honest -- but it must name the plugin, not read as
# the generic "Resource not found" that started this.
assert 'not installed' in response.get_json()['message'].lower()
class TestTheScriptRunner:
def test_it_returns_the_json_the_script_prints(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text(
'print(\'{"status": "success", "auth_url": "https://x"}\')\n',
encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert error is None
assert payload['auth_url'] == 'https://x'
def test_it_ignores_noise_before_the_json(self, tmp_path):
# An import warning or a library writing to stdout would otherwise
# make the last-line parse fail.
script = tmp_path / 'calendar_registration.py'
script.write_text(
'print("some library warning")\n'
'print(\'{"status": "success"}\')\n', encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert error is None and payload['status'] == 'success'
def test_it_passes_stdin_through(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text(
'import sys, json\n'
'print(json.dumps({"status": "success", "got": sys.stdin.read().strip()}))\n',
encoding='utf-8')
payload, _ = mod._run_calendar_registration(tmp_path, 'http://127.0.0.1/?code=abc')
assert payload['got'] == 'http://127.0.0.1/?code=abc'
def test_a_missing_script_is_reported(self, tmp_path):
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'script not found' in error.lower()
def test_output_that_is_not_json_is_reported_with_context(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text('import sys\nsys.stderr.write("boom\\n")\n', encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'no result' in error.lower()
assert 'boom' in error
class TestListingShape:
"""The picker reads cal.id, cal.summary and cal.primary."""
def _authenticate(self, client, monkeypatch, items):
creds = type('C', (), {'expired': False, 'refresh_token': None, 'valid': True})()
(client.plugin_dir / 'token.pickle').write_bytes(pickle.dumps({'x': 1}))
monkeypatch.setattr(mod.pickle if hasattr(mod, 'pickle') else pickle,
'loads', lambda *a, **k: creds, raising=False)
import types
fake_pickle = types.SimpleNamespace(load=lambda f: creds, dump=lambda *a: None)
# Callers pass a flat list of calendars; the API returns them wrapped
# in a page. One page is all these cases need -- TestPagination builds
# its own multi-page sequences.
pages = [{'items': items}]
state = {'i': 0}
def fake_list(**kwargs):
page = pages[min(state['i'], len(pages) - 1)]
state['i'] += 1
return types.SimpleNamespace(execute=lambda: page)
def fake_build(*args, **kwargs):
return types.SimpleNamespace(
calendarList=lambda: types.SimpleNamespace(list=fake_list))
real_import = __builtins__['__import__'] if isinstance(__builtins__, dict) \
else __builtins__.__import__
def fake_import(name, *args, **kwargs):
if name == 'pickle':
return fake_pickle
if name == 'google.auth.transport.requests':
return types.SimpleNamespace(Request=object)
if name == 'googleapiclient.discovery':
return types.SimpleNamespace(build=fake_build)
return real_import(name, *args, **kwargs)
monkeypatch.setattr('builtins.__import__', fake_import)
def test_it_returns_id_summary_and_primary(self, client, monkeypatch):
self._authenticate(client, monkeypatch, [
{'id': 'b@x', 'summary': 'Work'},
{'id': 'a@x', 'summary': 'Personal', 'primary': True},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['status'] == 'success'
assert {c['id'] for c in body['calendars']} == {'a@x', 'b@x'}
assert all(set(c) == {'id', 'summary', 'primary'} for c in body['calendars'])
def test_the_primary_calendar_comes_first(self, client, monkeypatch):
# Short list, but the one the user wants is almost always their own.
self._authenticate(client, monkeypatch, [
{'id': 'z@x', 'summary': 'Aardvarks'},
{'id': 'a@x', 'summary': 'Zebras', 'primary': True},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['calendars'][0]['id'] == 'a@x'
assert body['calendars'][0]['primary'] is True
def test_a_calendar_without_a_name_still_lists(self, client, monkeypatch):
self._authenticate(client, monkeypatch, [{'id': 'noname@x'}])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['calendars'][0]['summary'] == 'noname@x'
def test_entries_without_an_id_are_dropped(self, client, monkeypatch):
# Nothing could be selected by such a row, and the checkbox value
# would be undefined.
self._authenticate(client, monkeypatch, [{'summary': 'ghost'}, {'id': 'real@x'}])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert [c['id'] for c in body['calendars']] == ['real@x']
class TestPagination:
"""calendarList.list pages at 250 and defaults to 100."""
def _paged(self, client, monkeypatch, pages):
import types
creds = type('C', (), {'expired': False, 'refresh_token': None, 'valid': True})()
(client.plugin_dir / 'token.pickle').write_bytes(b'x')
state = {'i': 0}
seen = []
def fake_list(**kwargs):
seen.append(kwargs)
page = pages[min(state['i'], len(pages) - 1)]
state['i'] += 1
return types.SimpleNamespace(execute=lambda: page)
def fake_build(*args, **kwargs):
return types.SimpleNamespace(
calendarList=lambda: types.SimpleNamespace(list=fake_list))
real_import = __builtins__['__import__'] if isinstance(__builtins__, dict) \
else __builtins__.__import__
def fake_import(name, *args, **kwargs):
if name == 'pickle':
return types.SimpleNamespace(load=lambda f: creds, dump=lambda *a: None)
if name == 'google.auth.transport.requests':
return types.SimpleNamespace(Request=object)
if name == 'googleapiclient.discovery':
return types.SimpleNamespace(build=fake_build)
return real_import(name, *args, **kwargs)
monkeypatch.setattr('builtins.__import__', fake_import)
return seen
def test_every_page_is_collected(self, client, monkeypatch):
# Taking only the first page would hide calendars from the picker with
# nothing to say the list was cut short.
self._paged(client, monkeypatch, [
{'items': [{'id': 'a@x', 'summary': 'A'}], 'nextPageToken': 't1'},
{'items': [{'id': 'b@x', 'summary': 'B'}], 'nextPageToken': 't2'},
{'items': [{'id': 'c@x', 'summary': 'C'}]},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert [c['id'] for c in body['calendars']] == ['a@x', 'b@x', 'c@x']
def test_the_page_token_is_passed_back(self, client, monkeypatch):
seen = self._paged(client, monkeypatch, [
{'items': [{'id': 'a@x', 'summary': 'A'}], 'nextPageToken': 'tok'},
{'items': [{'id': 'b@x', 'summary': 'B'}]},
])
client.get('/api/v3/plugins/calendar/list-calendars')
assert seen[0]['pageToken'] is None
assert seen[1]['pageToken'] == 'tok'
assert all(k['maxResults'] == 250 for k in seen)
def test_a_looping_token_cannot_spin_forever(self, client, monkeypatch):
# Every page claims another follows.
self._paged(client, monkeypatch, [
{'items': [{'id': 'a@x', 'summary': 'A'}], 'nextPageToken': 'same'},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['status'] == 'success'
assert len(body['calendars']) <= mod._CALENDAR_LIST_MAX_PAGES
class TestDiagnosticsAreRedacted:
def test_script_stderr_is_redacted_on_the_way_out(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text(
'import sys\n'
'sys.stderr.write("boom client_secret=hunter2 more\\n")\n',
encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'hunter2' not in error, error
assert '<redacted>' in error, error
def test_a_failing_script_payload_is_redacted(self, client):
(client.plugin_dir / 'credentials.json').write_text('{}', encoding='utf-8')
(client.plugin_dir / 'calendar_registration.py').write_text(
'import json\n'
'print(json.dumps({"status": "error", '
'"message": "Failed: client_secret=topsecret"}))\n',
encoding='utf-8')
body = client.post('/api/v3/plugins/calendar/authenticate',
json={}).get_json()
assert body['status'] == 'error'
assert 'topsecret' not in json.dumps(body), body
assert '<redacted>' in body['message'], body
def test_an_unrunnable_script_is_reported_without_raw_exception_text(self,
tmp_path,
monkeypatch):
# OSError from the spawn carries the interpreter path and whatever the
# OS chose to say; it reaches the client through the redactor like
# everything else.
script = tmp_path / 'calendar_registration.py'
script.write_text('', encoding='utf-8')
def boom(*a, **k):
raise OSError("Exec format error: token=abcd1234 /usr/bin/python3")
monkeypatch.setattr(mod.subprocess, 'run', boom)
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'abcd1234' not in error, error
assert 'OSError' in error, error
def test_a_missing_google_library_is_reported_without_raw_exception_text(
self, client, monkeypatch):
(client.plugin_dir / 'token.pickle').write_bytes(b'x')
real_import = __builtins__['__import__'] if isinstance(__builtins__, dict) \
else __builtins__.__import__
def fake_import(name, *args, **kwargs):
if name.startswith('google'):
raise ImportError("No module named 'google' password=hunter2")
return real_import(name, *args, **kwargs)
monkeypatch.setattr('builtins.__import__', fake_import)
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert 'hunter2' not in json.dumps(body), body
assert 'requirements.txt' in body['message']
+33 -3
View File
@@ -16,6 +16,8 @@ from datetime import datetime, timedelta
sys.path.insert(0, str(Path(__file__).parent.parent)) sys.path.insert(0, str(Path(__file__).parent.parent))
from src.config_manager import ConfigManager from src.config_manager import ConfigManager
from src.web_interface.error_handler import describe_exception
from werkzeug.exceptions import HTTPException
from src.exceptions import ConfigError from src.exceptions import ConfigError
from src.plugin_system.plugin_manager import PluginManager from src.plugin_system.plugin_manager import PluginManager
from src.plugin_system.store_manager import PluginStoreManager from src.plugin_system.store_manager import PluginStoreManager
@@ -391,15 +393,42 @@ def internal_error(error):
import logging import logging
logger = logging.getLogger('web_interface') logger = logging.getLogger('web_interface')
logger.error("Internal server error", exc_info=True) logger.error("Internal server error", exc_info=True)
return jsonify({ payload = {
'status': 'error', 'status': 'error',
'error_code': 'INTERNAL_ERROR', 'error_code': 'INTERNAL_ERROR',
'message': 'An internal error occurred; see logs for details', 'message': 'An internal error occurred; see logs for details',
}), 500 }
# Flask hands the original exception over as `error.original_exception`
# when propagation is off; without it there is nothing to describe.
original = getattr(error, 'original_exception', None) or (
error if isinstance(error, BaseException) else None)
if original is not None:
payload['details'] = describe_exception(original)
return jsonify(payload), 500
@app.errorhandler(Exception) @app.errorhandler(Exception)
def handle_exception(error): def handle_exception(error):
"""Handle all unhandled exceptions.""" """Handle all unhandled exceptions.
Returning only "see logs for details" is fine until the logs are exactly
what you cannot reach. A device with failing storage answered every
endpoint with that sentence -- including the log viewer, because journalctl
could not be executed -- while the exception underneath said
`[Errno 5] Input/output error`. Naming the error costs nothing here and is
frequently the whole diagnosis, so include it alongside the log pointer.
"""
# Werkzeug's HTTPExceptions subclass Exception, so this catch-all sees
# them too and was reporting every 405, 400, 413 and 415 as a server-side
# UNKNOWN_ERROR 500. A GET on a POST-only route came back as "an error
# occurred" rather than "method not allowed", which tells the caller
# nothing and blames the wrong side. Hand those back as themselves.
if isinstance(error, HTTPException):
return jsonify({
'status': 'error',
'error_code': (error.name or 'HTTP_ERROR').upper().replace(' ', '_'),
'message': error.description,
}), error.code or 500
import logging import logging
logger = logging.getLogger('web_interface') logger = logging.getLogger('web_interface')
logger.error("Unhandled exception", exc_info=True) logger.error("Unhandled exception", exc_info=True)
@@ -407,6 +436,7 @@ def handle_exception(error):
'status': 'error', 'status': 'error',
'error_code': 'UNKNOWN_ERROR', 'error_code': 'UNKNOWN_ERROR',
'message': 'An error occurred; see logs for details', 'message': 'An error occurred; see logs for details',
'details': describe_exception(error),
}), 500 }), 500
# Captive portal redirect middleware # Captive portal redirect middleware
+319 -70
View File
@@ -22,6 +22,7 @@ logger = logging.getLogger(__name__)
from src.web_interface.api_helpers import success_response, error_response, validate_request_json from src.web_interface.api_helpers import success_response, error_response, validate_request_json
from src.web_interface.errors import ErrorCode from src.web_interface.errors import ErrorCode
from src.web_interface.secret_helpers import find_secret_fields, separate_secrets from src.web_interface.secret_helpers import find_secret_fields, separate_secrets
from src.web_interface.error_handler import describe_exception, redact_text
from src.plugin_system.operation_types import OperationType from src.plugin_system.operation_types import OperationType
from src.web_interface.validators import ( from src.web_interface.validators import (
validate_file_upload validate_file_upload
@@ -272,7 +273,7 @@ def get_main_config():
return jsonify({'status': 'success', 'data': config}) return jsonify({'status': 'success', 'data': config})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/config/schedule', methods=['GET']) @api_v3.route('/config/schedule', methods=['GET'])
def get_schedule_config(): def get_schedule_config():
@@ -290,9 +291,11 @@ def get_schedule_config():
return success_response(data=schedule_config) return success_response(data=schedule_config)
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return error_response( return error_response(
ErrorCode.CONFIG_LOAD_FAILED, ErrorCode.CONFIG_LOAD_FAILED,
"An error occurred; see logs for details", "An error occurred; see logs for details",
details=describe_exception(e),
status_code=500 status_code=500
) )
@@ -468,7 +471,7 @@ def save_schedule_config():
ErrorCode.CONFIG_SAVE_FAILED, ErrorCode.CONFIG_SAVE_FAILED,
"An error occurred; see logs for details", "An error occurred; see logs for details",
status_code=500 status_code=500, details=describe_exception(e)
) )
@api_v3.route('/config/dim-schedule', methods=['GET']) @api_v3.route('/config/dim-schedule', methods=['GET'])
@@ -516,14 +519,14 @@ def get_dim_schedule_config():
return error_response( return error_response(
ErrorCode.CONFIG_LOAD_FAILED, ErrorCode.CONFIG_LOAD_FAILED,
"An error occurred; see logs for details", "An error occurred; see logs for details",
status_code=500 status_code=500, details=describe_exception(e)
) )
except Exception as e: except Exception as e:
logging.error(f"[DIM SCHEDULE] Unexpected error loading config: {e}", exc_info=True) logging.error(f"[DIM SCHEDULE] Unexpected error loading config: {e}", exc_info=True)
return error_response( return error_response(
ErrorCode.CONFIG_LOAD_FAILED, ErrorCode.CONFIG_LOAD_FAILED,
"An error occurred; see logs for details", "An error occurred; see logs for details",
status_code=500 status_code=500, details=describe_exception(e)
) )
@api_v3.route('/config/dim-schedule', methods=['POST']) @api_v3.route('/config/dim-schedule', methods=['POST'])
@@ -687,7 +690,7 @@ def save_dim_schedule_config():
ErrorCode.CONFIG_SAVE_FAILED, ErrorCode.CONFIG_SAVE_FAILED,
"An error occurred; see logs for details", "An error occurred; see logs for details",
status_code=500 status_code=500, details=describe_exception(e)
) )
@api_v3.route('/config/main', methods=['POST']) @api_v3.route('/config/main', methods=['POST'])
@@ -793,7 +796,7 @@ def save_main_config():
'gpio_slowdown', 'rp1_rio', 'scan_mode', 'disable_hardware_pulsing', 'inverse_colors', 'show_refresh_rate', 'gpio_slowdown', 'rp1_rio', 'scan_mode', 'disable_hardware_pulsing', 'inverse_colors', 'show_refresh_rate',
'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz', 'use_short_date_format', 'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz', 'use_short_date_format',
'max_dynamic_duration_seconds', 'led_rgb_sequence', 'multiplexing', 'panel_type', 'max_dynamic_duration_seconds', 'led_rgb_sequence', 'multiplexing', 'panel_type',
'row_address_type', 'pixel_mapper_config'] 'row_address_type', 'pixel_mapper_config', 'orientation']
if any(k in data for k in display_fields): if any(k in data for k in display_fields):
if 'display' not in current_config: if 'display' not in current_config:
@@ -828,6 +831,11 @@ def save_main_config():
if 'pixel_mapper_config' in data and not isinstance(data['pixel_mapper_config'], str): if 'pixel_mapper_config' in data and not isinstance(data['pixel_mapper_config'], str):
return jsonify({'status': 'error', 'message': 'pixel_mapper_config must be a string (e.g. "U-mapper;Rotate:90" or empty)'}), 400 return jsonify({'status': 'error', 'message': 'pixel_mapper_config must be a string (e.g. "U-mapper;Rotate:90" or empty)'}), 400
# Validate orientation (physical mounting rotation; composed onto pixel_mapper_config at runtime)
ORIENTATION_ALLOWED = {'normal', '180'}
if 'orientation' in data and data['orientation'] not in ORIENTATION_ALLOWED:
return jsonify({'status': 'error', 'message': f"Invalid orientation '{data['orientation']}'. Allowed values: {', '.join(sorted(ORIENTATION_ALLOWED))}"}), 400
# Validate row_address_type # Validate row_address_type
if 'row_address_type' in data: if 'row_address_type' in data:
try: try:
@@ -841,7 +849,7 @@ def save_main_config():
for field in ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'hardware_mapping', 'scan_mode', for field in ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'hardware_mapping', 'scan_mode',
'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz', 'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz',
'led_rgb_sequence', 'multiplexing', 'panel_type', 'row_address_type', 'led_rgb_sequence', 'multiplexing', 'panel_type', 'row_address_type',
'pixel_mapper_config']: 'pixel_mapper_config', 'orientation']:
if field in data: if field in data:
if field in ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'scan_mode', if field in ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'scan_mode',
'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz', 'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz',
@@ -1314,7 +1322,7 @@ def save_main_config():
return error_response( return error_response(
ErrorCode.CONFIG_SAVE_FAILED, ErrorCode.CONFIG_SAVE_FAILED,
"An error occurred; see logs for details", "An error occurred; see logs for details",
status_code=500 status_code=500, details=describe_exception(e)
) )
@api_v3.route('/config/secrets', methods=['GET']) @api_v3.route('/config/secrets', methods=['GET'])
@@ -1328,7 +1336,7 @@ def get_secrets_config():
return jsonify({'status': 'success', 'data': config}) return jsonify({'status': 'success', 'data': config})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/config/raw/main', methods=['POST']) @api_v3.route('/config/raw/main', methods=['POST'])
def save_raw_main_config(): def save_raw_main_config():
@@ -1361,6 +1369,7 @@ def save_raw_main_config():
return error_response( return error_response(
ErrorCode.CONFIG_SAVE_FAILED, ErrorCode.CONFIG_SAVE_FAILED,
error_message, error_message,
details=describe_exception(e),
context={'config_path': e.config_path} if hasattr(e, 'config_path') and e.config_path else None, context={'config_path': e.config_path} if hasattr(e, 'config_path') and e.config_path else None,
status_code=500 status_code=500
@@ -1370,6 +1379,7 @@ def save_raw_main_config():
return error_response( return error_response(
ErrorCode.UNKNOWN_ERROR, ErrorCode.UNKNOWN_ERROR,
error_message, error_message,
details=describe_exception(e),
status_code=500 status_code=500
) )
@@ -1409,7 +1419,8 @@ def save_raw_secrets_config():
else: else:
error_message = 'An error occurred; see logs for details' error_message = 'An error occurred; see logs for details'
return jsonify({'status': 'error', 'message': error_message}), 500 return jsonify({'status': 'error', 'message': error_message,
'details': describe_exception(e)}), 500
@api_v3.route('/system/status', methods=['GET']) @api_v3.route('/system/status', methods=['GET'])
def get_system_status(): def get_system_status():
@@ -1497,7 +1508,7 @@ def get_system_status():
return jsonify({'status': 'success', 'data': status}) return jsonify({'status': 'success', 'data': status})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/health', methods=['GET']) @api_v3.route('/health', methods=['GET'])
def get_health(): def get_health():
@@ -1596,9 +1607,11 @@ def get_health():
return jsonify({'status': 'success', 'data': health_status}) return jsonify({'status': 'success', 'data': health_status})
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'message': 'An error occurred; see logs for details',
'details': describe_exception(e),
'data': {'status': 'unhealthy'} 'data': {'status': 'unhealthy'}
}), 500 }), 500
@@ -2368,7 +2381,7 @@ def get_display_current():
return jsonify({'status': 'success', 'data': display_data}) return jsonify({'status': 'success', 'data': display_data})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/display/on-demand/status', methods=['GET']) @api_v3.route('/display/on-demand/status', methods=['GET'])
def get_on_demand_status(): def get_on_demand_status():
@@ -2392,7 +2405,7 @@ def get_on_demand_status():
}) })
except Exception as exc: except Exception as exc:
logger.error('Error in get_on_demand_status', exc_info=True) logger.error('Error in get_on_demand_status', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(exc)}), 500
@api_v3.route('/display/on-demand/start', methods=['POST']) @api_v3.route('/display/on-demand/start', methods=['POST'])
def start_on_demand_display(): def start_on_demand_display():
@@ -2495,7 +2508,7 @@ def start_on_demand_display():
return jsonify({'status': 'success', 'data': response_data}) return jsonify({'status': 'success', 'data': response_data})
except Exception as exc: except Exception as exc:
logger.error('Error in start_on_demand_display', exc_info=True) logger.error('Error in start_on_demand_display', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(exc)}), 500
@api_v3.route('/display/on-demand/stop', methods=['POST']) @api_v3.route('/display/on-demand/stop', methods=['POST'])
def stop_on_demand_display(): def stop_on_demand_display():
@@ -2531,7 +2544,7 @@ def stop_on_demand_display():
}) })
except Exception as exc: except Exception as exc:
logger.error('Error in stop_on_demand_display', exc_info=True) logger.error('Error in stop_on_demand_display', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(exc)}), 500
@api_v3.route('/plugins/installed', methods=['GET']) @api_v3.route('/plugins/installed', methods=['GET'])
def get_installed_plugins(): def get_installed_plugins():
@@ -2679,7 +2692,7 @@ def get_installed_plugins():
return jsonify({'status': 'success', 'data': {'plugins': plugins}}) return jsonify({'status': 'success', 'data': {'plugins': plugins}})
except Exception as e: except Exception as e:
logger.error('Error in get_installed_plugins', exc_info=True) logger.error('Error in get_installed_plugins', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
def _installed_plugin_ids(): def _installed_plugin_ids():
"""Best-effort list of installed plugin IDs for the web process. """Best-effort list of installed plugin IDs for the web process.
@@ -2745,7 +2758,7 @@ def get_plugin_health():
}) })
except Exception as e: except Exception as e:
logger.error('Error in get_plugin_health', exc_info=True) logger.error('Error in get_plugin_health', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/health/<plugin_id>', methods=['GET']) @api_v3.route('/plugins/health/<plugin_id>', methods=['GET'])
def get_plugin_health_single(plugin_id): def get_plugin_health_single(plugin_id):
@@ -2770,7 +2783,7 @@ def get_plugin_health_single(plugin_id):
}) })
except Exception as e: except Exception as e:
logger.error('Error in get_plugin_health_single', exc_info=True) logger.error('Error in get_plugin_health_single', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/health/<plugin_id>/reset', methods=['POST']) @api_v3.route('/plugins/health/<plugin_id>/reset', methods=['POST'])
def reset_plugin_health(plugin_id): def reset_plugin_health(plugin_id):
@@ -2795,7 +2808,7 @@ def reset_plugin_health(plugin_id):
}) })
except Exception as e: except Exception as e:
logger.error('Error in reset_plugin_health', exc_info=True) logger.error('Error in reset_plugin_health', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/metrics', methods=['GET']) @api_v3.route('/plugins/metrics', methods=['GET'])
def get_plugin_metrics(): def get_plugin_metrics():
@@ -2835,7 +2848,7 @@ def get_plugin_metrics():
}) })
except Exception as e: except Exception as e:
logger.error('Error in get_plugin_metrics', exc_info=True) logger.error('Error in get_plugin_metrics', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/metrics/<plugin_id>', methods=['GET']) @api_v3.route('/plugins/metrics/<plugin_id>', methods=['GET'])
def get_plugin_metrics_single(plugin_id): def get_plugin_metrics_single(plugin_id):
@@ -2860,7 +2873,7 @@ def get_plugin_metrics_single(plugin_id):
}) })
except Exception as e: except Exception as e:
logger.error('Error in get_plugin_metrics_single', exc_info=True) logger.error('Error in get_plugin_metrics_single', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/metrics/<plugin_id>/reset', methods=['POST']) @api_v3.route('/plugins/metrics/<plugin_id>/reset', methods=['POST'])
def reset_plugin_metrics(plugin_id): def reset_plugin_metrics(plugin_id):
@@ -2885,7 +2898,7 @@ def reset_plugin_metrics(plugin_id):
}) })
except Exception as e: except Exception as e:
logger.error('Error in reset_plugin_metrics', exc_info=True) logger.error('Error in reset_plugin_metrics', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/limits/<plugin_id>', methods=['GET', 'POST']) @api_v3.route('/plugins/limits/<plugin_id>', methods=['GET', 'POST'])
def manage_plugin_limits(plugin_id): def manage_plugin_limits(plugin_id):
@@ -2940,7 +2953,7 @@ def manage_plugin_limits(plugin_id):
}) })
except Exception as e: except Exception as e:
logger.error('Error in manage_plugin_limits', exc_info=True) logger.error('Error in manage_plugin_limits', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/toggle', methods=['POST']) @api_v3.route('/plugins/toggle', methods=['POST'])
def toggle_plugin(): def toggle_plugin():
@@ -3949,7 +3962,7 @@ def install_plugin():
except Exception as e: except Exception as e:
logger.error('Error in install_plugin', exc_info=True) logger.error('Error in install_plugin', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/install-from-url', methods=['POST']) @api_v3.route('/plugins/install-from-url', methods=['POST'])
def install_plugin_from_url(): def install_plugin_from_url():
@@ -4004,7 +4017,7 @@ def install_plugin_from_url():
except Exception as e: except Exception as e:
logger.error('Error in install_plugin_from_url', exc_info=True) logger.error('Error in install_plugin_from_url', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/registry-from-url', methods=['POST']) @api_v3.route('/plugins/registry-from-url', methods=['POST'])
def get_registry_from_url(): def get_registry_from_url():
@@ -4036,7 +4049,7 @@ def get_registry_from_url():
except Exception as e: except Exception as e:
logger.error('Error in get_registry_from_url', exc_info=True) logger.error('Error in get_registry_from_url', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/saved-repositories', methods=['GET']) @api_v3.route('/plugins/saved-repositories', methods=['GET'])
def get_saved_repositories(): def get_saved_repositories():
@@ -4049,7 +4062,7 @@ def get_saved_repositories():
return jsonify({'status': 'success', 'data': {'repositories': repositories}}) return jsonify({'status': 'success', 'data': {'repositories': repositories}})
except Exception as e: except Exception as e:
logger.error('Error in get_saved_repositories', exc_info=True) logger.error('Error in get_saved_repositories', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/saved-repositories', methods=['POST']) @api_v3.route('/plugins/saved-repositories', methods=['POST'])
def add_saved_repository(): def add_saved_repository():
@@ -4080,7 +4093,7 @@ def add_saved_repository():
}), 400 }), 400
except Exception as e: except Exception as e:
logger.error('Error in add_saved_repository', exc_info=True) logger.error('Error in add_saved_repository', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/saved-repositories', methods=['DELETE']) @api_v3.route('/plugins/saved-repositories', methods=['DELETE'])
def remove_saved_repository(): def remove_saved_repository():
@@ -4110,7 +4123,7 @@ def remove_saved_repository():
}), 404 }), 404
except Exception as e: except Exception as e:
logger.error('Error in remove_saved_repository', exc_info=True) logger.error('Error in remove_saved_repository', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/store/list', methods=['GET']) @api_v3.route('/plugins/store/list', methods=['GET'])
def list_plugin_store(): def list_plugin_store():
@@ -4163,7 +4176,7 @@ def list_plugin_store():
return jsonify({'status': 'success', 'data': {'plugins': formatted_plugins}}) return jsonify({'status': 'success', 'data': {'plugins': formatted_plugins}})
except Exception as e: except Exception as e:
logger.error('Error in list_plugin_store', exc_info=True) logger.error('Error in list_plugin_store', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/store/github-status', methods=['GET']) @api_v3.route('/plugins/store/github-status', methods=['GET'])
def get_github_auth_status(): def get_github_auth_status():
@@ -4214,7 +4227,7 @@ def get_github_auth_status():
}) })
except Exception as e: except Exception as e:
logger.error('Error in get_github_auth_status', exc_info=True) logger.error('Error in get_github_auth_status', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/store/refresh', methods=['POST']) @api_v3.route('/plugins/store/refresh', methods=['POST'])
def refresh_plugin_store(): def refresh_plugin_store():
@@ -4241,7 +4254,7 @@ def refresh_plugin_store():
}) })
except Exception as e: except Exception as e:
logger.error('Error in refresh_plugin_store', exc_info=True) logger.error('Error in refresh_plugin_store', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
def deep_merge(base_dict, update_dict): def deep_merge(base_dict, update_dict):
""" """
@@ -5763,7 +5776,7 @@ def get_plugin_schema():
return jsonify({'status': 'success', 'data': {'schema': default_schema}}) return jsonify({'status': 'success', 'data': {'schema': default_schema}})
except Exception as e: except Exception as e:
logger.error('Error in get_plugin_schema', exc_info=True) logger.error('Error in get_plugin_schema', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/skins', methods=['GET']) @api_v3.route('/skins', methods=['GET'])
def list_skins(): def list_skins():
@@ -5798,9 +5811,9 @@ def list_skins():
'has_preview': bool(preview and (skin_dir / preview).is_file()), 'has_preview': bool(preview and (skin_dir / preview).is_file()),
}) })
return jsonify({'status': 'success', 'data': {'skins': payload}}) return jsonify({'status': 'success', 'data': {'skins': payload}})
except Exception: except Exception as e:
logger.error('Error in list_skins', exc_info=True) logger.error('Error in list_skins', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/config/reset', methods=['POST']) @api_v3.route('/plugins/config/reset', methods=['POST'])
def reset_plugin_config(): def reset_plugin_config():
@@ -5880,7 +5893,7 @@ def reset_plugin_config():
}) })
except Exception as e: except Exception as e:
logger.error('Error in reset_plugin_config', exc_info=True) logger.error('Error in reset_plugin_config', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/action', methods=['POST']) @api_v3.route('/plugins/action', methods=['POST'])
def execute_plugin_action(): def execute_plugin_action():
@@ -6140,7 +6153,7 @@ sys.exit(proc.returncode)
logger.error("Error executing action step 1", exc_info=True) logger.error("Error executing action step 1", exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details' 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500 }), 500
else: else:
# Simple script execution # Simple script execution
@@ -6190,7 +6203,7 @@ sys.exit(proc.returncode)
return jsonify({'status': 'error', 'message': 'Action timed out'}), 408 return jsonify({'status': 'error', 'message': 'Action timed out'}), 408
except Exception as e: except Exception as e:
logger.error('Error in execute_plugin_action', exc_info=True) logger.error('Error in execute_plugin_action', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/authenticate/spotify', methods=['POST']) @api_v3.route('/plugins/authenticate/spotify', methods=['POST'])
def authenticate_spotify(): def authenticate_spotify():
@@ -6323,12 +6336,12 @@ sys.exit(proc.returncode)
logger.error("Error getting Spotify auth URL", exc_info=True) logger.error("Error getting Spotify auth URL", exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details' 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500 }), 500
except Exception as e: except Exception as e:
logger.error('Error in authenticate_spotify', exc_info=True) logger.error('Error in authenticate_spotify', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/authenticate/ytm', methods=['POST']) @api_v3.route('/plugins/authenticate/ytm', methods=['POST'])
def authenticate_ytm(): def authenticate_ytm():
@@ -6378,7 +6391,7 @@ def authenticate_ytm():
return jsonify({'status': 'error', 'message': 'Authentication timed out'}), 408 return jsonify({'status': 'error', 'message': 'Authentication timed out'}), 408
except Exception as e: except Exception as e:
logger.error('Error in authenticate_ytm', exc_info=True) logger.error('Error in authenticate_ytm', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/catalog', methods=['GET']) @api_v3.route('/fonts/catalog', methods=['GET'])
def get_fonts_catalog(): def get_fonts_catalog():
@@ -6473,7 +6486,10 @@ def get_fonts_catalog():
return jsonify({'status': 'success', 'data': {'catalog': catalog}}) return jsonify({'status': 'success', 'data': {'catalog': catalog}})
except Exception as e: except Exception as e:
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 logger.error("%s failed", request.path, exc_info=True)
return jsonify({'status': 'error',
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)}), 500
@api_v3.route('/fonts/tokens', methods=['GET']) @api_v3.route('/fonts/tokens', methods=['GET'])
def get_font_tokens(): def get_font_tokens():
@@ -6492,7 +6508,7 @@ def get_font_tokens():
return jsonify({'status': 'success', 'data': {'tokens': tokens}}) return jsonify({'status': 'success', 'data': {'tokens': tokens}})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/overrides', methods=['GET']) @api_v3.route('/fonts/overrides', methods=['GET'])
def get_fonts_overrides(): def get_fonts_overrides():
@@ -6504,7 +6520,7 @@ def get_fonts_overrides():
return jsonify({'status': 'success', 'data': {'overrides': overrides}}) return jsonify({'status': 'success', 'data': {'overrides': overrides}})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/overrides', methods=['POST']) @api_v3.route('/fonts/overrides', methods=['POST'])
def save_fonts_overrides(): def save_fonts_overrides():
@@ -6518,7 +6534,7 @@ def save_fonts_overrides():
return jsonify({'status': 'success', 'message': 'Font overrides saved'}) return jsonify({'status': 'success', 'message': 'Font overrides saved'})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/overrides/<element_key>', methods=['DELETE']) @api_v3.route('/fonts/overrides/<element_key>', methods=['DELETE'])
def delete_font_override(element_key): def delete_font_override(element_key):
@@ -6528,7 +6544,7 @@ def delete_font_override(element_key):
return jsonify({'status': 'success', 'message': f'Font override for {element_key} deleted'}) return jsonify({'status': 'success', 'message': f'Font override for {element_key} deleted'})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/upload', methods=['POST']) @api_v3.route('/fonts/upload', methods=['POST'])
def upload_font(): def upload_font():
@@ -6593,7 +6609,7 @@ def upload_font():
}) })
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/preview', methods=['GET']) @api_v3.route('/fonts/preview', methods=['GET'])
@@ -6738,7 +6754,7 @@ def get_font_preview() -> tuple[Response, int] | Response:
}) })
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/<font_family>', methods=['DELETE']) @api_v3.route('/fonts/<font_family>', methods=['DELETE'])
@@ -6826,7 +6842,7 @@ def delete_font(font_family: str) -> tuple[Response, int] | Response:
}) })
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/assets/upload', methods=['POST']) @api_v3.route('/plugins/assets/upload', methods=['POST'])
@@ -6974,7 +6990,7 @@ def upload_plugin_asset():
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/of-the-day/json/upload', methods=['POST']) @api_v3.route('/plugins/of-the-day/json/upload', methods=['POST'])
def upload_of_the_day_json(): def upload_of_the_day_json():
@@ -7124,7 +7140,7 @@ def upload_of_the_day_json():
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/of-the-day/json/delete', methods=['POST']) @api_v3.route('/plugins/of-the-day/json/delete', methods=['POST'])
def delete_of_the_day_json(): def delete_of_the_day_json():
@@ -7171,7 +7187,7 @@ def delete_of_the_day_json():
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/<plugin_id>/static/<path:file_path>', methods=['GET']) @api_v3.route('/plugins/<plugin_id>/static/<path:file_path>', methods=['GET'])
def serve_plugin_static(plugin_id, file_path): def serve_plugin_static(plugin_id, file_path):
@@ -7217,7 +7233,7 @@ def serve_plugin_static(plugin_id, file_path):
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/calendar/upload-credentials', methods=['POST']) @api_v3.route('/plugins/calendar/upload-credentials', methods=['POST'])
@@ -7299,7 +7315,228 @@ def upload_calendar_credentials():
except Exception as e: except Exception as e:
logger.error('Error in upload_calendar_credentials', exc_info=True) logger.error('Error in upload_calendar_credentials', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
# calendarList.list pages at 250 entries maximum. Ten pages is far past any
# real account and exists only so a malformed nextPageToken cannot spin here.
_CALENDAR_LIST_MAX_PAGES = 10
def _calendar_plugin_dir() -> Optional[Path]:
"""Where the calendar plugin is installed, or None if it is not."""
if api_v3.plugin_manager:
plugin_dir = api_v3.plugin_manager.get_plugin_directory('calendar')
else:
plugin_dir = PROJECT_ROOT / 'plugins' / 'calendar'
if not plugin_dir:
return None
plugin_dir = Path(plugin_dir)
return plugin_dir if plugin_dir.exists() else None
def _run_calendar_registration(plugin_dir: Path, stdin_payload: str):
"""Run the plugin's OAuth script and return the JSON object it prints.
The script decides between web and terminal mode by whether stdin is a
tty, so it must be given a pipe. It emits one JSON object on stdout; the
last parsable line is taken, because an import warning or a library's
stderr redirection can land in front of it.
Returns (payload, error_message). Exactly one is None.
"""
script = plugin_dir / 'calendar_registration.py'
if not script.exists():
return None, 'Authentication script not found in the calendar plugin'
try:
result = subprocess.run( # nosec B603 - fixed script path inside the plugin dir
[sys.executable, str(script)],
input=stdin_payload,
capture_output=True,
text=True,
timeout=120,
cwd=str(plugin_dir),
)
except subprocess.TimeoutExpired:
return None, 'Authentication timed out after 120s'
except OSError as e:
logger.error('Could not run calendar_registration.py', exc_info=True)
return None, 'Could not run the authentication script: %s' % describe_exception(e)
for line in reversed((result.stdout or '').splitlines()):
line = line.strip()
if not line:
continue
try:
payload = json.loads(line)
except json.JSONDecodeError:
continue
if isinstance(payload, dict):
return payload, None
raw = (result.stderr or result.stdout or '').strip()
# The unredacted text goes to the log, where it is worth having in full.
# What comes back over HTTP is redacted: this is a script that handles
# OAuth client secrets, and its stderr can quote them.
if raw:
logger.error('calendar_registration.py failed (exit %s): %s',
result.returncode, raw)
return None, 'Authentication script produced no result%s' % (
': %s' % redact_text(raw) if raw else '')
@api_v3.route('/plugins/calendar/authenticate', methods=['POST'])
def authenticate_calendar():
"""Google OAuth for the calendar plugin, in the two steps it requires.
Step 1 (no body) returns the consent URL to open. Step 2 posts back the
URL Google redirected to -- it fails to load, because the redirect points
at a loopback address nothing is listening on, but the address bar carries
the authorization code -- and the script exchanges it for a token.
Two calls rather than one because the user has to visit Google in between.
The script persists the PKCE verifier from step 1 for step 2 to reuse; the
exchange fails with "Missing code verifier" otherwise.
"""
try:
plugin_dir = _calendar_plugin_dir()
if plugin_dir is None:
return jsonify({
'status': 'error',
'message': 'The calendar plugin is not installed'
}), 404
if not (plugin_dir / 'credentials.json').exists():
return jsonify({
'status': 'error',
'message': ('No credentials.json yet. Upload your Google OAuth '
'client file first (Step 1).')
}), 400
data = request.get_json(silent=True) or {}
redirect_url = (data.get('redirect_url') or data.get('code') or '').strip()
payload, error = _run_calendar_registration(plugin_dir, redirect_url)
if error:
return jsonify({'status': 'error', 'message': error}), 500
if payload.get('status') != 'success':
# The script's own diagnosis is more useful than anything that
# could be reconstructed here -- but it interpolates exceptions
# into its messages, so it reaches the client redacted and the
# original goes to the log.
logger.error('calendar authentication failed: %s', payload)
safe = dict(payload)
safe['message'] = redact_text(str(payload.get('message', '')
or 'Authentication failed'))
return jsonify(safe), 400
return jsonify(payload)
except Exception as e:
logger.error('Error in authenticate_calendar', exc_info=True)
return jsonify({'status': 'error',
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)}), 500
@api_v3.route('/plugins/calendar/list-calendars', methods=['GET'])
def list_calendar_calendars():
"""The calendars this account can see, for the config picker.
Reads the token the OAuth flow wrote rather than shelling out again: the
picker is used interactively and a subprocess per click is slower than the
API call it would be wrapping.
"""
try:
plugin_dir = _calendar_plugin_dir()
if plugin_dir is None:
return jsonify({
'status': 'error',
'message': 'The calendar plugin is not installed'
}), 404
token_file = plugin_dir / 'token.pickle'
if not token_file.exists():
return jsonify({
'status': 'error',
'message': ('Not authenticated with Google yet. Complete Step 2 '
'first, then load your calendars.')
}), 400
try:
import pickle
from google.auth.transport.requests import Request as GoogleRequest
from googleapiclient.discovery import build as build_google_service
except ImportError as e:
return jsonify({
'status': 'error',
# The name of the missing module is the whole diagnosis, but it
# arrives as an exception, so it goes through the redactor like
# any other -- an ImportError can quote a path.
'message': ('The Google API libraries are not installed. Install '
"the calendar plugin's requirements.txt. (%s)"
% describe_exception(e))
}), 500
with open(token_file, 'rb') as handle:
# Written only by this plugin's own OAuth flow, into its own
# directory, and read here exactly as the plugin itself reads it.
creds = pickle.load(handle) # nosec B301 - locally generated token
if creds and creds.expired and creds.refresh_token:
creds.refresh(GoogleRequest())
with open(token_file, 'wb') as handle:
pickle.dump(creds, handle)
os.chmod(token_file, 0o600)
if not creds or not creds.valid:
return jsonify({
'status': 'error',
'message': ('Stored Google credentials are no longer valid. '
'Run Step 2 again to re-authenticate.')
}), 400
service = build_google_service('calendar', 'v3', credentials=creds)
# calendarList.list returns 100 entries per page by default and caps at
# 250, handing back a nextPageToken when there are more. Taking only
# the first page would silently hide calendars from the picker, and the
# user would have no way to tell the list was truncated.
entries = []
page_token = None
for _ in range(_CALENDAR_LIST_MAX_PAGES):
response = service.calendarList().list(
maxResults=250, pageToken=page_token).execute()
entries.extend(response.get('items', []))
page_token = response.get('nextPageToken')
if not page_token:
break
else:
# 2500 calendars in, something is wrong with the account or the
# token is looping; show what was collected rather than spin.
logger.warning(
'calendarList paging stopped at %d pages with more remaining',
_CALENDAR_LIST_MAX_PAGES)
calendars = [{
'id': entry.get('id'),
# The picker labels each row with summary and falls back to the id
# only in its own display, so send something either way.
'summary': entry.get('summary') or entry.get('id'),
'primary': bool(entry.get('primary', False)),
} for entry in entries if entry.get('id')]
# Primary first, then alphabetically: the list is usually short but the
# one the user wants is almost always their own calendar.
calendars.sort(key=lambda c: (not c['primary'], c['summary'].lower()))
return jsonify({'status': 'success', 'calendars': calendars})
except Exception as e:
logger.error('Error in list_calendar_calendars', exc_info=True)
return jsonify({'status': 'error',
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)}), 500
@api_v3.route('/plugins/assets/delete', methods=['POST']) @api_v3.route('/plugins/assets/delete', methods=['POST'])
def delete_plugin_asset(): def delete_plugin_asset():
@@ -7342,7 +7579,7 @@ def delete_plugin_asset():
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/assets/list', methods=['GET']) @api_v3.route('/plugins/assets/list', methods=['GET'])
def list_plugin_assets(): def list_plugin_assets():
@@ -7370,7 +7607,7 @@ def list_plugin_assets():
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/display/current-status', methods=['GET']) @api_v3.route('/display/current-status', methods=['GET'])
def get_current_display_status(): def get_current_display_status():
@@ -7391,9 +7628,9 @@ def get_current_display_status():
'last_updated': None, 'last_updated': None,
} }
return jsonify({'status': 'success', 'data': state}) return jsonify({'status': 'success', 'data': state})
except Exception: except Exception as e:
logger.error('Error in get_current_display_status', exc_info=True) logger.error('Error in get_current_display_status', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/logs', methods=['GET']) @api_v3.route('/logs', methods=['GET'])
def get_logs(): def get_logs():
@@ -7432,9 +7669,11 @@ def get_logs():
'message': 'Timeout while fetching logs' 'message': 'Timeout while fetching logs'
}), 500 }), 500
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details' 'message': 'An error occurred; see logs for details',
'details': describe_exception(e)
}), 500 }), 500
# Multi-Display Sync Endpoints # Multi-Display Sync Endpoints
@@ -7499,9 +7738,11 @@ def get_wifi_status():
} }
}) })
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details' 'message': 'An error occurred; see logs for details',
'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/scan', methods=['GET']) @api_v3.route('/wifi/scan', methods=['GET'])
@@ -7623,7 +7864,7 @@ def connect_wifi():
logger.error("Error connecting to WiFi", exc_info=True) logger.error("Error connecting to WiFi", exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details' 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/disconnect', methods=['POST']) @api_v3.route('/wifi/disconnect', methods=['POST'])
@@ -7649,7 +7890,7 @@ def disconnect_wifi():
logger.error("Error disconnecting from WiFi", exc_info=True) logger.error("Error disconnecting from WiFi", exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details' 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/ap/enable', methods=['POST']) @api_v3.route('/wifi/ap/enable', methods=['POST'])
@@ -7674,9 +7915,11 @@ def enable_ap_mode():
'message': message 'message': message
}), 400 }), 400
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details' 'message': 'An error occurred; see logs for details',
'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/ap/disable', methods=['POST']) @api_v3.route('/wifi/ap/disable', methods=['POST'])
@@ -7699,9 +7942,11 @@ def disable_ap_mode():
'message': message 'message': message
}), 400 }), 400
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details' 'message': 'An error occurred; see logs for details',
'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/ap/auto-enable', methods=['GET']) @api_v3.route('/wifi/ap/auto-enable', methods=['GET'])
@@ -7720,9 +7965,11 @@ def get_auto_enable_ap_mode():
} }
}) })
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details' 'message': 'An error occurred; see logs for details',
'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/ap/auto-enable', methods=['POST']) @api_v3.route('/wifi/ap/auto-enable', methods=['POST'])
@@ -7752,9 +7999,11 @@ def set_auto_enable_ap_mode():
} }
}) })
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details' 'message': 'An error occurred; see logs for details',
'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/radio', methods=['GET']) @api_v3.route('/wifi/radio', methods=['GET'])
@@ -7774,7 +8023,7 @@ def get_wifi_radio():
logger.error("Error getting WiFi radio state", exc_info=True) logger.error("Error getting WiFi radio state", exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details' 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/radio', methods=['POST']) @api_v3.route('/wifi/radio', methods=['POST'])
@@ -7822,7 +8071,7 @@ def set_wifi_radio():
logger.error("Error setting WiFi radio state", exc_info=True) logger.error("Error setting WiFi radio state", exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details' 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/cache/list', methods=['GET']) @api_v3.route('/cache/list', methods=['GET'])
@@ -7847,7 +8096,7 @@ def list_cache_files():
}) })
except Exception as e: except Exception as e:
logger.error('Error in list_cache_files', exc_info=True) logger.error('Error in list_cache_files', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/cache/delete', methods=['POST']) @api_v3.route('/cache/delete', methods=['POST'])
def delete_cache_file(): def delete_cache_file():
@@ -7873,7 +8122,7 @@ def delete_cache_file():
}) })
except Exception as e: except Exception as e:
logger.error('Error in delete_cache_file', exc_info=True) logger.error('Error in delete_cache_file', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
# ============================================================================= # =============================================================================
@@ -0,0 +1,196 @@
/**
* Google OAuth Widget
*
* Step 2 of the calendar plugin's setup, between uploading the OAuth client
* file and picking calendars. Google will not let a headless device complete
* consent on its own, so the flow is necessarily two calls with a human in
* between:
*
* 1. POST /api/v3/plugins/calendar/authenticate with no body
* -> { auth_url } to open in a browser
* 2. the browser lands on a loopback address that fails to load; its URL
* carries the authorization code. POST it back as redirect_url
* -> the server exchanges it and writes token.pickle
*
* The failed page in step 2 is expected and is worth saying out loud, because
* it looks exactly like something went wrong.
*
* @module GoogleOAuthWidget
*/
(function () {
'use strict';
if (typeof window.LEDMatrixWidgets === 'undefined') {
console.error('[GoogleOAuthWidget] LEDMatrixWidgets registry not found. Load registry.js first.');
return;
}
const ENDPOINT = '/api/v3/plugins/calendar/authenticate';
window.LEDMatrixWidgets.register('google-oauth', {
name: 'Google OAuth Widget',
version: '1.0.0',
/**
* @param {HTMLElement} container
* @param {Object} config - schema config (unused)
* @param {*} value - unused; this widget stores nothing
* @param {Object} options - { fieldId, pluginId, name }
*/
render: function (container, config, value, options) {
const fieldId = options.fieldId;
// Nothing is stored in config by this step -- the result is
// token.pickle on the device -- but the form still expects a field.
const hidden = document.createElement('input');
hidden.type = 'hidden';
hidden.id = fieldId + '_hidden';
hidden.name = options.name;
hidden.value = value || '';
const startBtn = document.createElement('button');
startBtn.type = 'button';
startBtn.className = 'px-3 py-1.5 text-sm rounded-md bg-blue-600 hover:bg-blue-700 text-white';
startBtn.innerHTML = '<i class="fas fa-key"></i> Connect Google Account';
const status = document.createElement('p');
status.className = 'text-xs text-gray-400 mt-2';
// Every message this widget gives -- the consent link is ready,
// the exchange failed -- arrives here after an async call, so a
// screen reader is told nothing unless it is a live region.
status.setAttribute('role', 'status');
status.setAttribute('aria-live', 'polite');
const step2 = document.createElement('div');
step2.className = 'mt-3 hidden';
const link = document.createElement('a');
link.target = '_blank';
link.rel = 'noopener noreferrer';
link.className = 'text-blue-400 underline text-sm break-all';
link.textContent = 'Open the Google consent screen';
// Deliberately loud. After consent the browser is redirected to a
// loopback address nothing is listening on, so it lands on a
// browser error page -- which reads as a failure at exactly the
// moment the user has to act on it. Said quietly in grey it gets
// missed, and the flow looks broken when it is working.
const hint = document.createElement('div');
hint.className =
'mt-3 p-3 rounded-md border border-amber-500/60 bg-amber-500/10';
hint.innerHTML =
'<p class="text-sm text-amber-300 font-semibold">'
+ '<i class="fas fa-triangle-exclamation"></i> '
+ 'The next page will fail to load. That is expected.</p>'
+ '<p class="text-xs text-amber-200/90 mt-1">'
+ 'After you approve access, Google sends your browser to '
+ '<code>127.0.0.1</code>, where nothing is running \u2014 so you will see '
+ '"This site can\u2019t be reached" or similar. Nothing has gone wrong. '
+ 'Copy the <strong>entire address</strong> out of the address bar '
+ '(it contains <code>?code=...</code>) and paste it in the box below.</p>';
const codeInputId = fieldId + '_redirect_url';
const codeLabel = document.createElement('label');
codeLabel.className = 'block text-xs text-gray-300 mt-3';
codeLabel.textContent = 'Paste the address from that failed page here:';
// The label was visible but not associated, so the input still had
// no accessible name -- a placeholder is not one, and it vanishes
// on focus, which is exactly when the value is being pasted.
codeLabel.setAttribute('for', codeInputId);
const codeInput = document.createElement('input');
codeInput.type = 'text';
codeInput.id = codeInputId;
codeInput.placeholder = 'http://127.0.0.1/?code=...';
codeInput.className =
'mt-2 block w-full px-3 py-2 text-sm border border-gray-600 '
+ 'rounded-md bg-gray-800 text-gray-100';
const finishBtn = document.createElement('button');
finishBtn.type = 'button';
finishBtn.className = 'mt-2 px-3 py-1.5 text-sm rounded-md bg-green-600 hover:bg-green-700 text-white';
finishBtn.innerHTML = '<i class="fas fa-check"></i> Finish Authentication';
function say(message, kind) {
status.textContent = message;
status.className = 'text-xs mt-2 ' + (
kind === 'error' ? 'text-red-400'
: kind === 'success' ? 'text-green-400'
: 'text-gray-400');
}
function post(body) {
return fetch(ENDPOINT, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body || {})
}).then(function (r) {
return r.json().catch(function () {
// A non-JSON body here means the request never reached
// the handler -- worth saying so rather than "undefined".
return { status: 'error', message: 'Server returned ' + r.status };
});
});
}
startBtn.addEventListener('click', function () {
startBtn.disabled = true;
say('Requesting a consent link...');
post({}).then(function (data) {
startBtn.disabled = false;
if (data.status !== 'success' || !data.auth_url) {
say(data.message || 'Could not start authentication.', 'error');
return;
}
link.href = data.auth_url;
step2.classList.remove('hidden');
say(data.message || 'Open the link, approve, then paste the address back.');
}).catch(function (err) {
startBtn.disabled = false;
say('Request failed: ' + err.message, 'error');
});
});
finishBtn.addEventListener('click', function () {
const pasted = codeInput.value.trim();
if (!pasted) {
say('Paste the address your browser was redirected to.', 'error');
return;
}
finishBtn.disabled = true;
say('Exchanging the code with Google...');
post({ redirect_url: pasted }).then(function (data) {
finishBtn.disabled = false;
if (data.status !== 'success') {
say(data.message || 'Authentication failed.', 'error');
return;
}
say(data.message || 'Authenticated.', 'success');
step2.classList.add('hidden');
codeInput.value = '';
}).catch(function (err) {
finishBtn.disabled = false;
say('Request failed: ' + err.message, 'error');
});
});
step2.appendChild(link);
step2.appendChild(hint);
step2.appendChild(codeLabel);
step2.appendChild(codeInput);
step2.appendChild(finishBtn);
container.appendChild(hidden);
container.appendChild(startBtn);
container.appendChild(status);
container.appendChild(step2);
},
getValue: function (fieldId) {
const hidden = document.getElementById(fieldId + '_hidden');
return hidden ? hidden.value : '';
}
});
})();
+1
View File
@@ -987,6 +987,7 @@
<script src="{{ url_for('static', filename='v3/js/widgets/custom-feeds.js') }}" defer></script> <script src="{{ url_for('static', filename='v3/js/widgets/custom-feeds.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/array-table.js') }}" defer></script> <script src="{{ url_for('static', filename='v3/js/widgets/array-table.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/google-calendar-picker.js') }}" defer></script> <script src="{{ url_for('static', filename='v3/js/widgets/google-calendar-picker.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/google-oauth.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/day-selector.js') }}" defer></script> <script src="{{ url_for('static', filename='v3/js/widgets/day-selector.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/time-range.js') }}" defer></script> <script src="{{ url_for('static', filename='v3/js/widgets/time-range.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/time-picker.js') }}" defer></script> <script src="{{ url_for('static', filename='v3/js/widgets/time-picker.js') }}" defer></script>
@@ -117,6 +117,14 @@
</select> </select>
</div> </div>
<div class="form-group" id="setting-display-orientation" data-setting-key="display.hardware.orientation">
<label for="orientation" class="block text-sm font-medium text-gray-700">Panel Orientation{{ ui.help_tip('Rotates the rendered image to match how the panel is physically mounted.\nUse "Upside Down" if you flipped the panel 180° to move the Raspberry Pi / wiring to a more convenient side.', 'Panel Orientation') }}</label>
<select id="orientation" name="orientation" class="form-control">
<option value="normal" {% if main_config.display.hardware.get('orientation', 'normal') == "normal" %}selected{% endif %}>Normal</option>
<option value="180" {% if main_config.display.hardware.get('orientation', 'normal') == "180" %}selected{% endif %}>Upside Down (180°)</option>
</select>
</div>
<div class="form-group" id="setting-display-led_rgb_sequence" data-setting-key="display.hardware.led_rgb_sequence"> <div class="form-group" id="setting-display-led_rgb_sequence" data-setting-key="display.hardware.led_rgb_sequence">
<label for="led_rgb_sequence" class="block text-sm font-medium text-gray-700">LED RGB Sequence{{ ui.help_tip('Order the panel expects color channels in.\nChange this only if reds/greens/blues look swapped. Default: RGB.', 'LED RGB Sequence') }}</label> <label for="led_rgb_sequence" class="block text-sm font-medium text-gray-700">LED RGB Sequence{{ ui.help_tip('Order the panel expects color channels in.\nChange this only if reds/greens/blues look swapped. Default: RGB.', 'LED RGB Sequence') }}</label>
<select id="led_rgb_sequence" name="led_rgb_sequence" class="form-control"> <select id="led_rgb_sequence" name="led_rgb_sequence" class="form-control">
@@ -63,13 +63,13 @@
<!-- Getting Started checklist: non-gating, dismissible (localStorage), items <!-- Getting Started checklist: non-gating, dismissible (localStorage), items
auto-check from existing config/endpoints — no new persisted state. auto-check from existing config/endpoints — no new persisted state.
Known heuristic limits (acceptable, disclosed): values left at legitimate The timezone step is verified against the browser's own zone rather than
defaults (e.g. a user actually in Tampa) read as "not done". --> compared to the shipped default; see the data-check="timezone" block below
for why. -->
{% set _hw = main_config.display.hardware if main_config and main_config.display else {} %} {% set _hw = main_config.display.hardware if main_config and main_config.display else {} %}
{% set _hw_done = (_hw.rows or 0) > 0 and (_hw.cols or 0) > 0 and (_hw.chain_length or 0) > 0 %} {% set _hw_done = (_hw.rows or 0) > 0 and (_hw.cols or 0) > 0 and (_hw.chain_length or 0) > 0 %}
{% set _loc = main_config.location if main_config and main_config.location else {} %} {% set _loc = main_config.location if main_config and main_config.location else {} %}
{% set _loc_done = (main_config.timezone and main_config.timezone != 'America/New_York') {% set _tz = (main_config.timezone if main_config else '') or '' %}
or (_loc.city and _loc.city != 'Tampa') %}
<div id="getting-started-card" class="bg-blue-50 border border-blue-200 rounded-lg p-4 mb-4" style="display:none" role="region" aria-label="Getting started checklist"> <div id="getting-started-card" class="bg-blue-50 border border-blue-200 rounded-lg p-4 mb-4" style="display:none" role="region" aria-label="Getting started checklist">
<div class="flex items-start justify-between"> <div class="flex items-start justify-between">
<div class="flex-1"> <div class="flex-1">
@@ -78,8 +78,8 @@
<ul class="space-y-1 text-sm" id="getting-started-items"> <ul class="space-y-1 text-sm" id="getting-started-items">
<li><button type="button" class="gs-item text-left w-full" data-done="{{ '1' if _hw_done else '0' }}" data-tab="display"> <li><button type="button" class="gs-item text-left w-full" data-done="{{ '1' if _hw_done else '0' }}" data-tab="display">
<i class="far fa-square mr-2"></i>Set your panel size (Display tab)</button></li> <i class="far fa-square mr-2"></i>Set your panel size (Display tab)</button></li>
<li><button type="button" class="gs-item text-left w-full" data-done="{{ '1' if _loc_done else '0' }}" data-tab="general"> <li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="timezone" data-tz="{{ _tz }}" data-tab="general">
<i class="far fa-square mr-2"></i>Set your timezone and location (General tab)</button></li> <i class="far fa-square mr-2"></i>Set your timezone{% if _tz %} — currently {{ _tz }}{% if _loc.city %}, {{ _loc.city }}{% endif %}{% endif %} (General tab)<span data-gs-tz-note class="text-xs"></span></button></li>
<li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="installed" data-tab="plugins"> <li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="installed" data-tab="plugins">
<i class="far fa-square mr-2"></i>Install a plugin from the Plugin Store</button></li> <i class="far fa-square mr-2"></i>Install a plugin from the Plugin Store</button></li>
<li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="enabled" data-tab="plugins"> <li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="enabled" data-tab="plugins">
@@ -165,6 +165,91 @@
}); });
maybeAutoHide(); maybeAutoHide();
// Timezone: verified against the browser's own zone.
//
// This step used to tick when the saved timezone differed from the value
// config.template.json ships (America/New_York), with the saved city
// OR-ed in. Two things were wrong with that. "Differs from the default"
// answers "did somebody edit this?", but what the checklist needs to know
// is whether the value is RIGHT — so anyone who genuinely lives in the
// default zone could never satisfy it and the card nagged forever. And
// the city has no bearing on whether the timezone is set: because the two
// were OR-ed, saving a city ticked the step off with the timezone still
// wrong, which is the direction that actually breaks displays (event
// times render in the wrong zone).
//
// The browser already knows its zone, so compare against that: no new
// persisted state, no network, and it catches the reverse case too — a
// panel still set to the old zone after a move now stays unticked, where
// the old test ticked it the moment the value stopped being the default.
function sameZone(a, b) {
if (a === b) return true;
// Compare the wall-clock time each zone yields, not the identifiers:
// aliases (Asia/Calcutta vs Asia/Kolkata, Europe/Kiev vs Europe/Kyiv)
// name one zone and must not read as a mismatch.
//
// Sampled at three instants, all of which have to agree. Checking only
// now is not enough: America/New_York and America/Lima hold the same
// offset all winter, so a panel set to the wrong one of those would
// tick in January and then run an hour off from March. Mid-January and
// mid-July sit either side of DST in both hemispheres, so only zones
// that agree year-round match -- while Toronto still matches New York,
// which is right, since either renders the same times.
try {
var now = new Date();
var year = now.getUTCFullYear();
var instants = [now,
new Date(Date.UTC(year, 0, 15, 12)),
new Date(Date.UTC(year, 6, 15, 12))];
var stamp = function (tz, at) {
// Explicit numeric fields rather than dateStyle/timeStyle:
// those are late additions to Intl (Firefox shipped them in
// 91), and an implementation that does not know them ignores
// them and formats the date alone. That would compare
// New York, Chicago and Madrid as equal and tick the step for
// a timezone that is plainly wrong -- the exact failure this
// check exists to catch. These options have been in Intl
// since ECMA-402 v1.
return new Intl.DateTimeFormat('en-US', {
timeZone: tz, year: 'numeric', month: '2-digit',
day: '2-digit', hour: '2-digit', minute: '2-digit',
hour12: false
}).format(at);
};
for (var i = 0; i < instants.length; i++) {
if (stamp(a, instants[i]) !== stamp(b, instants[i])) {
return false;
}
}
return true;
} catch (e) {
// An unparseable zone in the config is worth surfacing, not hiding.
return false;
}
}
(function () {
var tzBtn = card.querySelector('[data-check="timezone"]');
if (!tzBtn) return;
var configured = tzBtn.dataset.tz || '';
if (!configured) return; // nothing saved yet: leave it open
var local = '';
try {
local = (Intl.DateTimeFormat().resolvedOptions().timeZone) || '';
} catch (e) {
return; // no Intl: leave it to the manual tick
}
if (!local) return;
if (sameZone(configured, local)) {
markDone(tzBtn);
return;
}
// Unticked on its own says "wrong" without saying why; name the zone
// the browser is in so the step is actionable.
var note = tzBtn.querySelector('[data-gs-tz-note]');
if (note) note.textContent = ' — this browser is in ' + local;
}());
// Plugin-derived states from the existing installed-plugins endpoint. // Plugin-derived states from the existing installed-plugins endpoint.
fetch('/api/v3/plugins/installed') fetch('/api/v3/plugins/installed')
.then(function (r) { return r.json(); }) .then(function (r) { return r.json(); })
@@ -815,7 +815,7 @@
<i class="fas fa-info-circle mr-1"></i> <i class="fas fa-info-circle mr-1"></i>
Changes in the file manager save immediately — no need to click Save Configuration. Changes in the file manager save immediately — no need to click Save Configuration.
</p> </p>
{% elif str_widget in ['text-input', 'textarea', 'select-dropdown', 'toggle-switch', 'radio-group', 'date-picker', 'time-picker', 'slider', 'color-picker', 'email-input', 'url-input', 'password-input', 'font-selector', 'file-upload-single', 'plugin-file-manager'] %} {% elif str_widget in ['text-input', 'textarea', 'select-dropdown', 'toggle-switch', 'radio-group', 'date-picker', 'time-picker', 'slider', 'color-picker', 'email-input', 'url-input', 'password-input', 'font-selector', 'file-upload-single', 'plugin-file-manager', 'google-oauth'] %}
{# Render widget container #} {# Render widget container #}
<div id="{{ field_id }}_container" class="{{ str_widget }}-container"></div> <div id="{{ field_id }}_container" class="{{ str_widget }}-container"></div>
<script> <script>