Compare commits

..
Author SHA1 Message Date
ChuckBuildsandClaude Opus 5 838f0b9d8a fix(calendar): redact script diagnostics, and page the calendar list
Three findings from CodeRabbit, all valid.

Raw subprocess output was being returned to the client -- the script's
stderr on one path, and its own error payload on another. CodeQL flagged
the same line. That script handles OAuth client secrets and interpolates
exceptions into its messages, so either could carry a secret or a path.
Both now go to the log unredacted, where they are worth having in full,
and reach the client through a redactor.

That redactor already existed inside describe_exception, which only
takes exceptions. Split out as redact_text: an exception is not the only
thing worth returning, and a subprocess's stderr is just as capable of
quoting a token.

calendarList.list returns 100 entries per page by default, caps at 250,
and hands back a nextPageToken when there are more. Reading one page
would have hidden calendars from the picker with nothing to say the list
was cut short. It now pages, asking for 250 at a time, bounded at ten
pages so a malformed token cannot spin.

And a test helper was a lambda where ruff wants a def.

The five new tests fail against the previous commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5
2026-08-13 09:34:26 -04:00
ChuckBuildsandClaude Opus 5 1b9ecc0f19 fix(calendar): one input for the auth code, and a louder warning
Two things reported after testing the flow.

There were two boxes and no way to tell which to use. The config
template's string branch dispatches widgets from an allow-list of names,
and anything missing from it falls through to a plain input type=text --
so the field rendered both the widget's own box and a stray one for the
same key. google-oauth is now on that list, which is all the widget ever
needed to render in place of the fallback rather than beside it.

And the warning that the redirect page fails to load was small grey text
under a link, which is where it is least likely to be read. It is now an
amber callout that leads with "The next page will fail to load. That is
expected." The failure lands at exactly the moment the user has to act
on it, and it looks precisely like the flow breaking rather than
working. The paste box is labelled too, rather than relying on a
placeholder that vanishes on focus.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5
2026-08-13 09:12:56 -04:00
ChuckBuildsandClaude Opus 5 7685e94ca5 fix(calendar): implement the OAuth and calendar-listing endpoints
The plugin's config advertised a three-step setup and only step 1
existed. Step 3's picker fetched
/api/v3/plugins/calendar/list-calendars, which was never registered, so
Flask fell through to the global 404 handler and the user saw "Resource
not found" -- a message that names nothing and points nowhere. Step 2
had no endpoint at all, so even a working picker would have found no
token to list with.

Two routes, following the pattern the spotify and ytm plugins already
use for their own auth scripts:

  POST /plugins/calendar/authenticate    two-step Google OAuth
  GET  /plugins/calendar/list-calendars  calendars for the picker

The authenticate route drives calendar_registration.py, which the plugin
already ships and which was written expressly for this -- it reads a
redirect URL on stdin and prints one JSON object. It takes two calls
because a human has to visit Google in between; the script persists the
PKCE verifier from the first call for the second, without which the
exchange fails with "Missing code verifier".

The listing route reads the token directly rather than shelling out
again: the picker is interactive and a subprocess per click is slower
than the API call it would wrap. It refreshes an expired token in place,
sorts the primary calendar first, and drops entries with no id, which
could not be selected anyway.

Both name the plugin when it is not installed, rather than reproducing
the anonymous 404 that started this.

Verified against the live Google API on the dev rig: HTTP 200 with the
account's real calendars.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5
2026-08-13 08:27:47 -04:00
bb1a1671ec fix(cache): make the ttl parameter actually control expiry (#450)
CacheManager.set(key, data, ttl=...) stored the number and no read path
ever consulted it. Expiry came from a max_age inferred from substrings
in the key -- "live", "odds", "stock" -- so all 52 callers passing a ttl
were writing a value that did nothing. The docstring said so outright:
"stored for compatibility but expiration is still controlled via max_age
when reading". It is easier to read that as a note than as a defect,
which is presumably how it survived.

Both cache layers already hold the record when they decide, so each now
prefers an explicit ttl and falls back to max_age when there is none.
The caller that wrote the record knows what its data is; a substring
guess is a reasonable default for records that never said, and a poor
override for records that did.

Measured against a device's real cache of 8,875 entries carrying a ttl,
the inferred and intended values disagreed nearly everywhere:

    stocks    max_age  600  vs ttl    1800   4903 entries
    news      max_age 3600  vs ttl     600   1770 entries
    odds      max_age 1800  vs ttl    3600   1301 entries
    images    max_age  300  vs ttl 2592000     20 entries

In every case the ttl matches what the plugin plainly intended: stock
quotes cached for half an hour rather than ten minutes, headlines
refreshed every ten minutes rather than hourly, bird photographs that
never change kept for a month rather than five minutes.

Two things make this safe to land now. No sports_live entry carries a
ttl at all -- the live-score path does not use set(ttl=) -- so live
freshness is untouched, which matters with a season two weeks out. And
replaying the change against that real cache, 997 currently-expired
entries become live while not one live entry becomes expired, so there
is no invalidation spike on deploy.


Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 14:14:25 -04:00
8159afca43 fix(odds): stop a stalled ESPN taking the whole plugin update with it (#449)
Odds are fetched per live game from inside SportsLive.update(), with
show_odds defaulting on, and the plugin executor kills an operation at
30s. The odds request timeout was also 30s, so a single stalled request
consumed the entire budget and the update carrying every game's score
was killed.

Out of season that is invisible: preseason week 1 returns one game. A
Sunday slate is around sixteen, so the odds of at least one slow request
rise sharply just as the cost of losing the update does.

Shorten the request timeout to 5s, and after a network failure skip the
network for 60s. The timeout alone is not enough -- sixteen consecutive
5s timeouts still blow through -- and when ESPN is unreachable it is
unreachable for the whole slate, so the first failure already answers
the question for the rest of the pass.

    before: one stalled request = 30s = the entire budget
    after : 5s, the rest of the slate skipped, retry after 60s

The stale-cache fallback is unchanged: the cache is consulted before any
of this, and the failing request still falls back to it.

An earlier version of this branch also jittered the cache TTL to stagger
expiry across a slate. That has been dropped: CacheManager.set() stores
ttl for compatibility but the read path expires entries by a per-type
max_age (1800s for odds), so the jitter was inert. Making the read path
honour a per-entry ttl is a real fix but changes a contract 48 plugin
call sites already rely on, which is not a change to make two weeks
before the season.


Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 13:56:53 -04:00
44f59ede07 fix(web): say what actually went wrong instead of "unknown" (#448)
* fix(web): say what actually went wrong instead of "unknown"

Every failing endpoint returned "An error occurred; see logs for
details" and nothing else. That is survivable until the logs are the
thing you cannot reach: a device whose SD card was failing answered the
restart action, /system/status and /logs with that same sentence -- the
log viewer included, because journalctl could not be executed -- while
the exception underneath said

    [Errno 5] Input/output error: 'systemctl'

which names the fault outright. The only endpoint that helped was
/health, and only because it happens to pass a subprocess's stderr
through. Diagnosis came down to guessing which endpoint leaked something.

Add describe_exception(), returning "TypeName: message" on one line, and
populate the `details` field that the response schema has always had and
nothing ever filled. The type alone carries information -- a bare
PermissionError says more than any generic sentence.

Exception text is not automatically safe to echo: a requests error
quotes the URL it failed on, and plugins that authenticate by query
string put their key there. Credential values are redacted while the
parameter name is kept, since knowing which credential was involved is
part of the diagnosis. Length is capped and newlines collapsed so a
parser's context cannot flood a JSON field.

Nine handlers in api_v3 bound the exception and never used it, so the
promised log entry was never written either -- "see logs for details"
was false, not merely unhelpful. Those now log with a traceback and
carry the detail. The other 60 already logged and are unchanged; they
can adopt the helper as they are touched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(web): redact auth headers and URL userinfo, and cover every handler

Three review findings.

The sanitizer missed two credential shapes that requests puts in its
exception text verbatim: `Authorization: Bearer <token>` and
`https://user:password@host`. Both would have gone straight into a
response. The auth-scheme name and the username are kept -- they say
which credential and whose without being the secret.

The AST test only asked whether *something* had been logged, so a
`logger.info("failed")` satisfied it while discarding the exception just
as completely. It now requires an error-level record carrying exc_info
and `describe_exception()` called on the handler's own bound exception.

Enforcing that revealed the first cut had scoped itself wrongly. I had
converted the nine handlers that logged nothing and left the sixty that
logged, reasoning their detail was at least in the journal. But
/system/status is one of the sixty, and on the failing device it told me
nothing -- the journal was exactly what could not be read. Splitting
them left most of the diagnostic surface unhelpful for the case this
change exists for, so all sixty-nine now carry the detail.

Two handlers had no bound exception name, and three passed the message
through a variable rather than a literal; both shapes needed doing by
hand. Full suite: 2383 passed, one pre-existing unrelated failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(web): stop reporting client errors as server faults

Werkzeug's HTTPExceptions subclass Exception, so the catch-all handler
saw them too and turned every 405, 400, 413 and 415 into a 500
UNKNOWN_ERROR. A GET on a POST-only route answered "an error occurred;
see logs for details", which tells the caller nothing and blames the
wrong side -- found while probing a device whose POST-only config
endpoints did exactly that.

Hand HTTPExceptions back as themselves, with their own status and
description. A genuine server fault still reports as one, with the
detail this branch adds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(web): redact any auth scheme, and require the detail in the response

Two review findings.

The auth-header pattern listed Bearer, Basic, Digest and Token, so
`Authorization: ApiKey SECRET` or `Negotiate SECRET` went to the client
intact. A fixed list silently leaks whatever it does not name, and
plugin APIs invent their own schemes, so match any scheme name and keep
it while redacting the credential.

The AST test accepted a describe_exception(e) call anywhere in the
handler, which a handler could satisfy by computing the detail and
dropping it before returning the generic message. It now requires the
call inside every return expression, which is where it has to be to
reach the caller.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 13:56:16 -04:00
12 changed files with 1370 additions and 74 deletions
+16
View File
@@ -112,6 +112,22 @@ class DiskCache:
record_ts = None
now = time.time()
# An explicit per-entry ttl wins over the caller's max_age. The
# caller that wrote the record knows what its data is; max_age is
# inferred from substrings in the key ("live", "odds", "stock") and
# is only a fallback for records that never said. Until now the ttl
# was stored and ignored, so `set(key, data, ttl=...)` did nothing
# at all -- 48 plugin call sites and 4 in the core were writing a
# number no read path consulted.
effective_max_age = max_age
if isinstance(record, dict):
stored_ttl = record.get('ttl')
if isinstance(stored_ttl, (int, float)) and not isinstance(stored_ttl, bool) \
and stored_ttl >= 0:
effective_max_age = stored_ttl
max_age = effective_max_age
# max_age=None means "never expires" (mirrors MemoryCache and the
# cache_manager docstring). Guard it explicitly — otherwise the
# comparison below raises TypeError and the record is treated as a
+10
View File
@@ -57,6 +57,16 @@ class MemoryCache:
if timestamp is None:
return None
# An explicit per-entry ttl wins over the caller's max_age, matching
# DiskCache. max_age is inferred from substrings in the key and is
# only a fallback for records that did not say what they wanted.
record = self._cache[key]
if isinstance(record, dict):
stored_ttl = record.get('ttl')
if isinstance(stored_ttl, (int, float)) and not isinstance(stored_ttl, bool) \
and stored_ttl >= 0:
max_age = stored_ttl
# Check expiration
if max_age is not None and (now - timestamp) > max_age:
# Expired - remove it
+4 -2
View File
@@ -594,8 +594,10 @@ class CacheManager:
Args:
key: Cache key
data: Data to cache
ttl: Optional time-to-live in seconds (stored for compatibility but
expiration is still controlled via max_age when reading)
ttl: Time-to-live in seconds for this entry. Takes precedence over
the max_age a reader would otherwise apply, which is inferred
from the key and is only a fallback for entries that did not
say. Omit it to keep that inferred behaviour.
"""
cache_data = {
'data': data,
+92
View File
@@ -4,6 +4,7 @@ Centralized error handling for web interface.
Provides helpers for consistent error responses across API endpoints.
"""
import re
from typing import Any, Optional
from flask import jsonify
@@ -16,6 +17,97 @@ from src.logging_config import get_logger
logger = get_logger(__name__)
# Credentials that turn up inside exception text. A requests error quotes the
# URL it failed on, and plugins that authenticate by query string put their key
# there, so echoing an exception verbatim can hand out an API key. Redact the
# value, keep the parameter name -- knowing *which* credential was involved is
# part of the diagnosis.
_REDACT_CREDENTIAL = re.compile(
r'((?:api[_-]?key|access[_-]?token|auth|apikey|key|passwd|password|pwd|'
r'secret|sig|signature|token)["\']?\s*[=:]\s*["\']?)([^\s&"\'<>,}]+)',
re.IGNORECASE,
)
# `Authorization: <scheme> <credential>`. The scheme name is kept because it
# says which kind of credential failed; the credential goes. Any scheme
# matches, not a fixed list: ApiKey, Negotiate, NTLM, AWS4-HMAC-SHA256 and
# whatever a plugin's API invents next are all credentials, and a list would
# silently leak the ones nobody thought of. Not covered by the generic pattern
# above, whose value part stops at whitespace and so would keep the credential
# once a space follows the scheme.
_REDACT_AUTH_HEADER = re.compile(
r'((?:proxy-)?authorization["\']?\s*[=:]\s*["\']?\s*'
r'(?:[A-Za-z][\w.+-]*[ \t]+)?)' # optional scheme name, kept
r'([^\s,"\'<>}]+)', # the credential, redacted
re.IGNORECASE,
)
# Credentials embedded in a URL: https://user:password@host. requests quotes
# the full URL in its exceptions, so this is a realistic leak. The username is
# kept -- it identifies which account failed without being the secret.
_REDACT_URL_USERINFO = re.compile(r'([a-z][a-z0-9+.-]*://[^/\s:@]+:)([^/\s@]+)(@)',
re.IGNORECASE)
# Long enough for an errno string with a path, short enough not to dump a
# parser's worth of context into a JSON field.
_MAX_DETAIL_LENGTH = 400
def describe_exception(exc: BaseException,
max_length: int = _MAX_DETAIL_LENGTH) -> str:
"""
One-line, safe-to-return description of an exception.
The generic "an error occurred; see logs for details" tells a user nothing
and, when the failure is bad enough, the logs are unreachable too: a device
whose storage was failing returned that message from every endpoint
*including* the log viewer, because journalctl could not be executed. The
underlying `[Errno 5] Input/output error` named the fault immediately.
Returns "TypeName: message", credentials redacted and length capped. The
type alone is worth carrying -- a bare PermissionError says more than any
generic sentence.
Args:
exc: The exception to describe
max_length: Truncate beyond this many characters
Returns:
A single-line description, never empty
"""
message = str(exc).strip()
text = f"{type(exc).__name__}: {message}" if message else type(exc).__name__
return redact_text(text, max_length)
def redact_text(text: str, max_length: int = _MAX_DETAIL_LENGTH) -> str:
"""Make arbitrary text safe to hand back over HTTP.
Split out of describe_exception because exceptions are not the only thing
worth returning: a subprocess's stderr, or a message a helper script
printed, is just as useful to a user and just as capable of carrying a
token or a password in it.
Args:
text: The text to redact
max_length: Truncate beyond this many characters
Returns:
A single line, credentials replaced, length capped.
"""
text = text or ''
# Order matters: the URL and header forms are more specific than the
# generic key=value pattern, which would otherwise chew the scheme.
text = _REDACT_URL_USERINFO.sub(r'\1<redacted>\3', text)
text = _REDACT_AUTH_HEADER.sub(r'\1<redacted>', text)
text = _REDACT_CREDENTIAL.sub(r'\1<redacted>', text)
# Collapse newlines/tabs so the detail stays one line in a JSON field.
text = ' '.join(text.split())
if len(text) > max_length:
text = text[:max_length - 1].rstrip() + ''
return text
def create_error_response(
error_code: ErrorCode,
message: str,
+130
View File
@@ -0,0 +1,130 @@
"""Tests that a per-entry ttl actually controls expiry.
Regression under test: `CacheManager.set(key, data, ttl=...)` stored the value
and no read path ever consulted it. Expiry came from a `max_age` inferred from
substrings in the key ("live", "odds", "stock"), so every caller passing `ttl=`
-- 48 sites across the plugins and 4 in the core -- was writing a number that
did nothing. The old docstring admitted as much: "stored for compatibility but
expiration is still controlled via max_age when reading".
Measured against a real device's cache (8,873 entries carrying a ttl), the
inferred value and the intended one disagreed almost everywhere:
stocks max_age 600 vs ttl 1800 4903 entries
news max_age 3600 vs ttl 600 1770 entries
odds max_age 1800 vs ttl 3600 1301 entries
images max_age 300 vs ttl 2592000 20 entries
No `sports_live` entry carries a ttl, so live scores keep their inferred
30-second freshness either way.
"""
import time
import pytest
from src.cache.memory_cache import MemoryCache
from src.cache.disk_cache import DiskCache
@pytest.fixture
def disk(tmp_path):
return DiskCache(cache_dir=str(tmp_path))
def _record(ttl=None, age=0.0):
rec = {"data": {"v": 1}, "timestamp": time.time() - age}
if ttl is not None:
rec["ttl"] = ttl
return rec
class TestDiskCacheHonoursTtl:
def test_ttl_longer_than_max_age_keeps_the_entry(self, disk):
# The odds case: written wanting an hour, expired at 30 minutes.
disk.set("odds_espn_football_nfl_401", _record(ttl=3600, age=1900))
assert disk.get("odds_espn_football_nfl_401", max_age=1800) is not None
def test_ttl_shorter_than_max_age_expires_the_entry(self, disk):
# The news case: written wanting 10 minutes, kept for an hour.
disk.set("news_NHL_1", _record(ttl=600, age=900))
assert disk.get("news_NHL_1", max_age=3600) is None
def test_without_a_ttl_max_age_still_applies(self, disk):
disk.set("plain_key", _record(age=400))
assert disk.get("plain_key", max_age=300) is None
disk.set("plain_key2", _record(age=100))
assert disk.get("plain_key2", max_age=300) is not None
def test_a_fresh_entry_within_its_ttl_survives(self, disk):
disk.set("k", _record(ttl=600, age=10))
assert disk.get("k", max_age=30) is not None
def test_ttl_zero_expires_immediately(self, disk):
# 0 means zero seconds, not "forever" -- max_age=None is how a caller
# asks for no expiry.
disk.set("k", _record(ttl=0, age=1))
assert disk.get("k", max_age=99999) is None
@pytest.mark.parametrize("bad", ["600", None, True, False, -5, {"a": 1}])
def test_a_nonsense_ttl_falls_back_to_max_age(self, disk, bad):
# Including bools: True is an int in Python and must not become a 1s ttl.
rec = _record(age=400)
rec["ttl"] = bad
disk.set("k_%s" % type(bad).__name__, rec)
assert disk.get("k_%s" % type(bad).__name__, max_age=300) is None
class TestMemoryCacheHonoursTtl:
def test_ttl_longer_than_max_age_keeps_the_entry(self):
m = MemoryCache()
m.set("k", _record(ttl=3600))
m._timestamps["k"] = time.time() - 1900
assert m.get("k", max_age=1800) is not None
def test_ttl_shorter_than_max_age_expires_the_entry(self):
m = MemoryCache()
m.set("k", _record(ttl=600))
m._timestamps["k"] = time.time() - 900
assert m.get("k", max_age=3600) is None
def test_without_a_ttl_max_age_still_applies(self):
m = MemoryCache()
m.set("k", _record())
m._timestamps["k"] = time.time() - 400
assert m.get("k", max_age=300) is None
def test_both_layers_agree(self, tmp_path):
"""A record must not be live in one layer and expired in the other."""
rec = _record(ttl=3600, age=1900)
d = DiskCache(cache_dir=str(tmp_path))
d.set("k", rec)
m = MemoryCache()
m.set("k", rec)
m._timestamps["k"] = rec["timestamp"]
assert (d.get("k", max_age=1800) is not None) == (m.get("k", max_age=1800) is not None)
class TestEndToEnd:
def test_set_then_get_respects_the_ttl(self, tmp_path, monkeypatch):
"""The behaviour a caller of CacheManager.set(ttl=...) expects."""
from src.cache_manager import CacheManager
cm = CacheManager()
cm._disk_cache_component = DiskCache(cache_dir=str(tmp_path))
cm._memory_cache_component = MemoryCache()
cm.set("odds_espn_football_nfl_401", {"spread": 6.5}, ttl=3600)
# Age the stored record past the inferred max_age for odds (1800s) but
# within the ttl the caller asked for.
path = cm._disk_cache_component.get_cache_path("odds_espn_football_nfl_401")
import json
rec = json.load(open(path))
rec["timestamp"] = time.time() - 1900
json.dump(rec, open(path, "w"))
cm._memory_cache_component.clear() if hasattr(
cm._memory_cache_component, "clear") else None
got = cm.get_with_auto_strategy("odds_espn_football_nfl_401")
assert got is not None, "the ttl the caller asked for was ignored"
+248
View File
@@ -0,0 +1,248 @@
"""Tests for surfacing the underlying error in web responses.
Regression under test: every failing endpoint returned "An error occurred; see
logs for details" and nothing else. On a device whose storage was failing that
sentence came back from the restart action, from /system/status, and from
/logs -- the log viewer itself -- because journalctl could not be executed. The
exception underneath said `[Errno 5] Input/output error: 'systemctl'`, which
names the fault outright, and nine handlers were discarding it entirely rather
than even logging it.
"""
import pytest
from src.web_interface.error_handler import describe_exception
class TestDescribeException:
def test_names_the_type_and_message(self):
detail = describe_exception(OSError(5, "Input/output error", "systemctl"))
assert detail == "OSError: [Errno 5] Input/output error: 'systemctl'"
def test_the_reported_failure_is_legible(self):
# The whole point: this string is the diagnosis.
assert "Input/output error" in describe_exception(
OSError(5, "Input/output error", "systemctl"))
def test_a_bare_exception_still_names_its_type(self):
# A PermissionError with no message still says more than "unknown".
assert describe_exception(PermissionError()) == "PermissionError"
assert describe_exception(Exception()) == "Exception"
def test_message_is_kept_when_present(self):
assert describe_exception(ValueError("bad port")) == "ValueError: bad port"
class TestCredentialRedaction:
"""Exception text quotes URLs, and plugins authenticate by query string."""
@pytest.mark.parametrize("secret_text,leaked", [
("failed: https://api.x.com/v1?api_key=SEC123&city=Tampa", "SEC123"),
("token=abcdef123456 was rejected", "abcdef123456"),
("connect failed password=hunter2", "hunter2"),
("GET /?access_token=zzz999", "zzz999"),
('{"secret": "topsecret"}', "topsecret"),
# requests quotes the URL it failed on, and both of these forms turn
# up in real client exceptions.
("401 for https://user:hunter2@example.com/api", "hunter2"),
("headers: {'Authorization': 'Bearer eyJ.SECRET.sig'}", "eyJ.SECRET.sig"),
("Authorization: Basic dXNlcjpwYXNzd29yZA==", "dXNlcjpwYXNzd29yZA=="),
("Proxy-Authorization: Bearer ptok999", "ptok999"),
# Any scheme, not a fixed list -- a list silently leaks whatever it
# does not name, and plugin APIs invent their own.
("Authorization: ApiKey SECRET123", "SECRET123"),
("Authorization: Negotiate YIIZnegotiateblob", "YIIZnegotiateblob"),
("Authorization: NTLM TlRMTVNTUAAB", "TlRMTVNTUAAB"),
("authorization: barecredential", "barecredential"),
])
def test_credentials_never_reach_the_response(self, secret_text, leaked):
detail = describe_exception(RuntimeError(secret_text))
assert leaked not in detail
assert "<redacted>" in detail
def test_the_parameter_name_survives_redaction(self):
# Knowing *which* credential was involved is part of the diagnosis.
detail = describe_exception(RuntimeError("https://x/y?api_key=SEC123"))
assert "api_key" in detail
def test_unknown_schemes_keep_their_name(self):
for scheme in ("ApiKey", "Negotiate", "NTLM", "AWS4-HMAC-SHA256"):
detail = describe_exception(
RuntimeError("Authorization: %s SECRETVALUE" % scheme))
assert scheme in detail, detail
assert "SECRETVALUE" not in detail, detail
def test_auth_scheme_and_username_survive(self):
# Which kind of credential, and whose, without the credential itself.
assert "Bearer" in describe_exception(
RuntimeError("Authorization: Bearer eyJ.SECRET.sig"))
assert "user" in describe_exception(
RuntimeError("https://user:hunter2@example.com"))
def test_non_secret_context_is_preserved(self):
detail = describe_exception(RuntimeError("https://api.x.com/v1?city=Tampa"))
assert "city=Tampa" in detail
assert "<redacted>" not in detail
class TestBounds:
def test_long_messages_are_truncated(self):
detail = describe_exception(ValueError("x" * 5000))
assert len(detail) <= 400
def test_newlines_are_collapsed_to_one_line(self):
detail = describe_exception(ValueError("line one\nline two\tthree"))
assert "\n" not in detail and "\t" not in detail
assert detail == "ValueError: line one line two three"
def test_custom_length_is_honoured(self):
assert len(describe_exception(ValueError("y" * 500), max_length=50)) <= 50
class TestHandlersCarryDetail:
"""The response shape callers actually see."""
def test_no_api_v3_handler_discards_its_exception(self):
"""Every generic-message handler must log a traceback and return detail.
Nine of them bound `e` and never used it, so the promised log entry was
never written either. Checking merely that *something* was logged is
too weak -- a `logger.info("failed")` would satisfy it while throwing
the exception away just as completely, so this asserts the two things
that actually make the failure diagnosable: an error-level record with
the traceback, and the sanitized detail in the response.
"""
import ast
src = open("web_interface/blueprints/api_v3.py").read()
tree = ast.parse(src)
generic = "An error occurred; see logs for details"
def logs_a_traceback(handler):
"""An error/exception-level log call carrying exc_info."""
for call in [n for n in ast.walk(handler) if isinstance(n, ast.Call)]:
func = call.func
if not isinstance(func, ast.Attribute):
continue
if func.attr == "exception": # implies exc_info
return True
if func.attr not in ("error", "critical"):
continue
if any(kw.arg == "exc_info" and getattr(kw.value, "value", False) is True
for kw in call.keywords):
return True
return False
def describes_this_exception(node, bound):
"""A describe_exception(<bound>) call anywhere under `node`."""
for call in [n for n in ast.walk(node) if isinstance(n, ast.Call)]:
if not (isinstance(call.func, ast.Name)
and call.func.id == "describe_exception"):
continue
if bound is None:
return True # bare `except:` cannot name it; accept
if any(isinstance(a, ast.Name) and a.id == bound
for a in call.args):
return True
return False
def returns_the_detail(handler):
"""The detail must be inside what the handler actually returns.
Looking anywhere in the handler is too weak: a handler could
compute describe_exception(e), drop it on the floor, and return the
generic message with no details field, while still passing. So the
call has to appear within a `return` expression.
"""
returns = [n for n in ast.walk(handler) if isinstance(n, ast.Return)]
if not returns:
return False
return all(describes_this_exception(r, handler.name) for r in returns)
offenders = []
for h in [n for n in ast.walk(tree) if isinstance(n, ast.ExceptHandler)]:
seg = ast.get_source_segment(src, h) or ""
if generic not in seg:
continue
missing = []
if not logs_a_traceback(h):
missing.append("error-level log with exc_info")
if not returns_the_detail(h):
missing.append("describe_exception(e) in the response")
if missing:
offenders.append((h.lineno, missing))
assert not offenders, (
"handlers returning the generic message without %s: %r"
% ("both a traceback log and the detail", offenders))
def test_client_errors_keep_their_own_status(self):
"""A 405 must not be reported as a server-side UNKNOWN_ERROR.
Werkzeug's HTTPExceptions subclass Exception, so the catch-all saw them
too: a GET on a POST-only route came back 500 "an error occurred",
which tells the caller nothing and blames the wrong side. Found while
probing a device whose POST-only config endpoints answered every GET
with UNKNOWN_ERROR.
"""
from flask import Flask, jsonify
from werkzeug.exceptions import HTTPException
app = Flask(__name__)
@app.errorhandler(Exception)
def handle(error):
if isinstance(error, HTTPException):
return jsonify({
"status": "error",
"error_code": (error.name or "HTTP_ERROR").upper().replace(" ", "_"),
"message": error.description,
}), error.code or 500
return jsonify({
"status": "error",
"error_code": "UNKNOWN_ERROR",
"message": "An error occurred; see logs for details",
"details": describe_exception(error),
}), 500
@app.route("/only-post", methods=["POST"])
def only_post():
return jsonify({"ok": True})
@app.route("/boom")
def boom():
raise OSError(5, "Input/output error", "systemctl")
client = app.test_client()
resp = client.get("/only-post")
assert resp.status_code == 405, "a wrong method must stay a 405"
assert resp.get_json()["error_code"] == "METHOD_NOT_ALLOWED"
# A genuine server fault still reports as one, with its detail.
resp = client.get("/boom")
assert resp.status_code == 500
assert "Input/output error" in resp.get_json()["details"]
def test_global_handler_reports_the_underlying_error(self):
from flask import Flask, jsonify
app = Flask(__name__)
@app.errorhandler(Exception)
def handle(error):
return jsonify({
"status": "error",
"error_code": "UNKNOWN_ERROR",
"message": "An error occurred; see logs for details",
"details": describe_exception(error),
}), 500
@app.route("/boom")
def boom():
raise OSError(5, "Input/output error", "systemctl")
client = app.test_client()
body = client.get("/boom").get_json()
assert body["error_code"] == "UNKNOWN_ERROR"
assert "Input/output error" in body["details"]
@@ -0,0 +1,344 @@
"""Tests the calendar plugin's OAuth and calendar-listing endpoints.
The plugin's config UI advertised a three-step setup, but only step 1 existed
on the server. Step 3's picker fetched /api/v3/plugins/calendar/list-calendars,
which was never registered, so Flask fell through to the global 404 handler and
the user saw "Resource not found" — with nothing to say which resource. Step 2
had no endpoint either, and no field in the schema at all, even though the
plugin ships calendar_registration.py written expressly for a web-driven
two-step flow.
These cover the two new routes: that they exist, that they fail with something
actionable rather than a bare 404, and that the shapes the widgets consume are
what the server actually sends.
"""
import json
import pickle
import sys
from pathlib import Path
import pytest
project_root = Path(__file__).parent.parent.parent
sys.path.insert(0, str(project_root))
from web_interface.blueprints import api_v3 as mod # noqa: E402
@pytest.fixture
def client(monkeypatch, tmp_path):
"""A test client whose calendar plugin lives in tmp_path."""
from flask import Flask
plugin_dir = tmp_path / 'calendar'
plugin_dir.mkdir()
app = Flask(__name__)
app.register_blueprint(mod.api_v3, url_prefix='/api/v3')
app.config['TESTING'] = True
monkeypatch.setattr(mod, '_calendar_plugin_dir', lambda: plugin_dir)
with app.test_client() as c:
c.plugin_dir = plugin_dir
yield c
@pytest.fixture
def uninstalled(monkeypatch):
from flask import Flask
app = Flask(__name__)
app.register_blueprint(mod.api_v3, url_prefix='/api/v3')
app.config['TESTING'] = True
monkeypatch.setattr(mod, '_calendar_plugin_dir', lambda: None)
with app.test_client() as c:
yield c
class TestTheRoutesExistAtAll:
"""The original bug: the URLs the widgets call were not registered."""
def test_list_calendars_is_routed(self, client):
response = client.get('/api/v3/plugins/calendar/list-calendars')
# Reaching the handler is the whole point; what it then says about
# missing setup is TestItSaysWhatIsWrong's business.
assert response.status_code != 404, "still unrouted"
assert response.get_json()['message'] != 'Resource not found'
def test_authenticate_is_routed(self, client):
response = client.post('/api/v3/plugins/calendar/authenticate', json={})
assert response.status_code != 404, "still unrouted"
assert response.get_json()['message'] != 'Resource not found'
def test_both_urls_match_what_the_widgets_request(self):
# The widgets hardcode these; a rename on either side reintroduces the
# original bug silently.
picker = Path(project_root) / 'web_interface/static/v3/js/widgets/google-calendar-picker.js'
oauth = Path(project_root) / 'web_interface/static/v3/js/widgets/google-oauth.js'
assert '/api/v3/plugins/calendar/list-calendars' in picker.read_text(encoding='utf-8')
assert '/api/v3/plugins/calendar/authenticate' in oauth.read_text(encoding='utf-8')
source = (Path(project_root) / 'web_interface/blueprints/api_v3.py').read_text(encoding='utf-8')
assert "'/plugins/calendar/list-calendars'" in source
assert "'/plugins/calendar/authenticate'" in source
def test_the_oauth_widget_is_dispatched_not_rendered_as_a_text_box(self):
# The string branch of the config template dispatches on an allow-list
# of widget names; anything missing from it silently falls through to a
# plain <input type="text">. That produced two boxes on the calendar
# page -- the widget's own, and a stray one for the same field -- and
# no way to tell which to paste into.
template = (Path(project_root)
/ 'web_interface/templates/v3/partials/plugin_config.html'
).read_text(encoding='utf-8')
allow_list_line = [ln for ln in template.splitlines()
if "str_widget in [" in ln]
assert allow_list_line, "the string widget allow-list moved"
assert "'google-oauth'" in allow_list_line[0], allow_list_line[0]
def test_the_widget_script_is_served(self):
base = (Path(project_root) / 'web_interface/templates/v3/base.html'
).read_text(encoding='utf-8')
assert 'widgets/google-oauth.js' in base
def test_the_failed_page_is_called_out_loudly(self):
# The loopback redirect lands on a browser error page at exactly the
# moment the user has to act. In small grey text it gets missed and the
# flow reads as broken while it is working.
widget = (Path(project_root)
/ 'web_interface/static/v3/js/widgets/google-oauth.js'
).read_text(encoding='utf-8')
assert 'expected' in widget.lower()
assert 'amber' in widget, "the warning is not visually distinguished"
class TestItSaysWhatIsWrong:
def test_listing_without_a_token_asks_for_step_2(self, client):
response = client.get('/api/v3/plugins/calendar/list-calendars')
assert response.status_code == 400
body = response.get_json()
assert body['status'] == 'error'
assert 'step 2' in body['message'].lower(), body['message']
def test_authenticating_without_credentials_asks_for_step_1(self, client):
response = client.post('/api/v3/plugins/calendar/authenticate', json={})
assert response.status_code == 400
assert 'step 1' in response.get_json()['message'].lower()
def test_an_uninstalled_plugin_says_so(self, uninstalled):
for response in (
uninstalled.get('/api/v3/plugins/calendar/list-calendars'),
uninstalled.post('/api/v3/plugins/calendar/authenticate', json={}),
):
assert response.status_code == 404
# A 404 here is honest -- but it must name the plugin, not read as
# the generic "Resource not found" that started this.
assert 'not installed' in response.get_json()['message'].lower()
class TestTheScriptRunner:
def test_it_returns_the_json_the_script_prints(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text(
'print(\'{"status": "success", "auth_url": "https://x"}\')\n',
encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert error is None
assert payload['auth_url'] == 'https://x'
def test_it_ignores_noise_before_the_json(self, tmp_path):
# An import warning or a library writing to stdout would otherwise
# make the last-line parse fail.
script = tmp_path / 'calendar_registration.py'
script.write_text(
'print("some library warning")\n'
'print(\'{"status": "success"}\')\n', encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert error is None and payload['status'] == 'success'
def test_it_passes_stdin_through(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text(
'import sys, json\n'
'print(json.dumps({"status": "success", "got": sys.stdin.read().strip()}))\n',
encoding='utf-8')
payload, _ = mod._run_calendar_registration(tmp_path, 'http://127.0.0.1/?code=abc')
assert payload['got'] == 'http://127.0.0.1/?code=abc'
def test_a_missing_script_is_reported(self, tmp_path):
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'script not found' in error.lower()
def test_output_that_is_not_json_is_reported_with_context(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text('import sys\nsys.stderr.write("boom\\n")\n', encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'no result' in error.lower()
assert 'boom' in error
class TestListingShape:
"""The picker reads cal.id, cal.summary and cal.primary."""
def _authenticate(self, client, monkeypatch, items):
creds = type('C', (), {'expired': False, 'refresh_token': None, 'valid': True})()
(client.plugin_dir / 'token.pickle').write_bytes(pickle.dumps({'x': 1}))
monkeypatch.setattr(mod.pickle if hasattr(mod, 'pickle') else pickle,
'loads', lambda *a, **k: creds, raising=False)
import types
fake_pickle = types.SimpleNamespace(load=lambda f: creds, dump=lambda *a: None)
pages = items if isinstance(items, list) and items and isinstance(items[0], dict) \
else items
if isinstance(pages, list):
pages = [{'items': pages}]
state = {'i': 0}
def fake_list(**kwargs):
page = pages[min(state['i'], len(pages) - 1)]
state['i'] += 1
return types.SimpleNamespace(execute=lambda: page)
def fake_build(*args, **kwargs):
return types.SimpleNamespace(
calendarList=lambda: types.SimpleNamespace(list=fake_list))
real_import = __builtins__['__import__'] if isinstance(__builtins__, dict) \
else __builtins__.__import__
def fake_import(name, *args, **kwargs):
if name == 'pickle':
return fake_pickle
if name == 'google.auth.transport.requests':
return types.SimpleNamespace(Request=object)
if name == 'googleapiclient.discovery':
return types.SimpleNamespace(build=fake_build)
return real_import(name, *args, **kwargs)
monkeypatch.setattr('builtins.__import__', fake_import)
def test_it_returns_id_summary_and_primary(self, client, monkeypatch):
self._authenticate(client, monkeypatch, [
{'id': 'b@x', 'summary': 'Work'},
{'id': 'a@x', 'summary': 'Personal', 'primary': True},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['status'] == 'success'
assert {c['id'] for c in body['calendars']} == {'a@x', 'b@x'}
assert all(set(c) == {'id', 'summary', 'primary'} for c in body['calendars'])
def test_the_primary_calendar_comes_first(self, client, monkeypatch):
# Short list, but the one the user wants is almost always their own.
self._authenticate(client, monkeypatch, [
{'id': 'z@x', 'summary': 'Aardvarks'},
{'id': 'a@x', 'summary': 'Zebras', 'primary': True},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['calendars'][0]['id'] == 'a@x'
assert body['calendars'][0]['primary'] is True
def test_a_calendar_without_a_name_still_lists(self, client, monkeypatch):
self._authenticate(client, monkeypatch, [{'id': 'noname@x'}])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['calendars'][0]['summary'] == 'noname@x'
def test_entries_without_an_id_are_dropped(self, client, monkeypatch):
# Nothing could be selected by such a row, and the checkbox value
# would be undefined.
self._authenticate(client, monkeypatch, [{'summary': 'ghost'}, {'id': 'real@x'}])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert [c['id'] for c in body['calendars']] == ['real@x']
class TestPagination:
"""calendarList.list pages at 250 and defaults to 100."""
def _paged(self, client, monkeypatch, pages):
import types
creds = type('C', (), {'expired': False, 'refresh_token': None, 'valid': True})()
(client.plugin_dir / 'token.pickle').write_bytes(b'x')
state = {'i': 0}
seen = []
def fake_list(**kwargs):
seen.append(kwargs)
page = pages[min(state['i'], len(pages) - 1)]
state['i'] += 1
return types.SimpleNamespace(execute=lambda: page)
def fake_build(*args, **kwargs):
return types.SimpleNamespace(
calendarList=lambda: types.SimpleNamespace(list=fake_list))
real_import = __builtins__['__import__'] if isinstance(__builtins__, dict) \
else __builtins__.__import__
def fake_import(name, *args, **kwargs):
if name == 'pickle':
return types.SimpleNamespace(load=lambda f: creds, dump=lambda *a: None)
if name == 'google.auth.transport.requests':
return types.SimpleNamespace(Request=object)
if name == 'googleapiclient.discovery':
return types.SimpleNamespace(build=fake_build)
return real_import(name, *args, **kwargs)
monkeypatch.setattr('builtins.__import__', fake_import)
return seen
def test_every_page_is_collected(self, client, monkeypatch):
# Taking only the first page would hide calendars from the picker with
# nothing to say the list was cut short.
self._paged(client, monkeypatch, [
{'items': [{'id': 'a@x', 'summary': 'A'}], 'nextPageToken': 't1'},
{'items': [{'id': 'b@x', 'summary': 'B'}], 'nextPageToken': 't2'},
{'items': [{'id': 'c@x', 'summary': 'C'}]},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert [c['id'] for c in body['calendars']] == ['a@x', 'b@x', 'c@x']
def test_the_page_token_is_passed_back(self, client, monkeypatch):
seen = self._paged(client, monkeypatch, [
{'items': [{'id': 'a@x', 'summary': 'A'}], 'nextPageToken': 'tok'},
{'items': [{'id': 'b@x', 'summary': 'B'}]},
])
client.get('/api/v3/plugins/calendar/list-calendars')
assert seen[0]['pageToken'] is None
assert seen[1]['pageToken'] == 'tok'
assert all(k['maxResults'] == 250 for k in seen)
def test_a_looping_token_cannot_spin_forever(self, client, monkeypatch):
# Every page claims another follows.
self._paged(client, monkeypatch, [
{'items': [{'id': 'a@x', 'summary': 'A'}], 'nextPageToken': 'same'},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['status'] == 'success'
assert len(body['calendars']) <= mod._CALENDAR_LIST_MAX_PAGES
class TestDiagnosticsAreRedacted:
def test_script_stderr_is_redacted_on_the_way_out(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text(
'import sys\n'
'sys.stderr.write("boom client_secret=hunter2 more\\n")\n',
encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'hunter2' not in error, error
assert '<redacted>' in error, error
def test_a_failing_script_payload_is_redacted(self, client):
(client.plugin_dir / 'credentials.json').write_text('{}', encoding='utf-8')
(client.plugin_dir / 'calendar_registration.py').write_text(
'import json\n'
'print(json.dumps({"status": "error", '
'"message": "Failed: client_secret=topsecret"}))\n',
encoding='utf-8')
body = client.post('/api/v3/plugins/calendar/authenticate',
json={}).get_json()
assert body['status'] == 'error'
assert 'topsecret' not in json.dumps(body), body
assert '<redacted>' in body['message'], body
+33 -3
View File
@@ -16,6 +16,8 @@ from datetime import datetime, timedelta
sys.path.insert(0, str(Path(__file__).parent.parent))
from src.config_manager import ConfigManager
from src.web_interface.error_handler import describe_exception
from werkzeug.exceptions import HTTPException
from src.exceptions import ConfigError
from src.plugin_system.plugin_manager import PluginManager
from src.plugin_system.store_manager import PluginStoreManager
@@ -391,15 +393,42 @@ def internal_error(error):
import logging
logger = logging.getLogger('web_interface')
logger.error("Internal server error", exc_info=True)
return jsonify({
payload = {
'status': 'error',
'error_code': 'INTERNAL_ERROR',
'message': 'An internal error occurred; see logs for details',
}), 500
}
# Flask hands the original exception over as `error.original_exception`
# when propagation is off; without it there is nothing to describe.
original = getattr(error, 'original_exception', None) or (
error if isinstance(error, BaseException) else None)
if original is not None:
payload['details'] = describe_exception(original)
return jsonify(payload), 500
@app.errorhandler(Exception)
def handle_exception(error):
"""Handle all unhandled exceptions."""
"""Handle all unhandled exceptions.
Returning only "see logs for details" is fine until the logs are exactly
what you cannot reach. A device with failing storage answered every
endpoint with that sentence -- including the log viewer, because journalctl
could not be executed -- while the exception underneath said
`[Errno 5] Input/output error`. Naming the error costs nothing here and is
frequently the whole diagnosis, so include it alongside the log pointer.
"""
# Werkzeug's HTTPExceptions subclass Exception, so this catch-all sees
# them too and was reporting every 405, 400, 413 and 415 as a server-side
# UNKNOWN_ERROR 500. A GET on a POST-only route came back as "an error
# occurred" rather than "method not allowed", which tells the caller
# nothing and blames the wrong side. Hand those back as themselves.
if isinstance(error, HTTPException):
return jsonify({
'status': 'error',
'error_code': (error.name or 'HTTP_ERROR').upper().replace(' ', '_'),
'message': error.description,
}), error.code or 500
import logging
logger = logging.getLogger('web_interface')
logger.error("Unhandled exception", exc_info=True)
@@ -407,6 +436,7 @@ def handle_exception(error):
'status': 'error',
'error_code': 'UNKNOWN_ERROR',
'message': 'An error occurred; see logs for details',
'details': describe_exception(error),
}), 500
# Captive portal redirect middleware
+307 -68
View File
@@ -22,6 +22,7 @@ logger = logging.getLogger(__name__)
from src.web_interface.api_helpers import success_response, error_response, validate_request_json
from src.web_interface.errors import ErrorCode
from src.web_interface.secret_helpers import find_secret_fields, separate_secrets
from src.web_interface.error_handler import describe_exception, redact_text
from src.plugin_system.operation_types import OperationType
from src.web_interface.validators import (
validate_file_upload
@@ -272,7 +273,7 @@ def get_main_config():
return jsonify({'status': 'success', 'data': config})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/config/schedule', methods=['GET'])
def get_schedule_config():
@@ -290,9 +291,11 @@ def get_schedule_config():
return success_response(data=schedule_config)
except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return error_response(
ErrorCode.CONFIG_LOAD_FAILED,
"An error occurred; see logs for details",
details=describe_exception(e),
status_code=500
)
@@ -468,7 +471,7 @@ def save_schedule_config():
ErrorCode.CONFIG_SAVE_FAILED,
"An error occurred; see logs for details",
status_code=500
status_code=500, details=describe_exception(e)
)
@api_v3.route('/config/dim-schedule', methods=['GET'])
@@ -516,14 +519,14 @@ def get_dim_schedule_config():
return error_response(
ErrorCode.CONFIG_LOAD_FAILED,
"An error occurred; see logs for details",
status_code=500
status_code=500, details=describe_exception(e)
)
except Exception as e:
logging.error(f"[DIM SCHEDULE] Unexpected error loading config: {e}", exc_info=True)
return error_response(
ErrorCode.CONFIG_LOAD_FAILED,
"An error occurred; see logs for details",
status_code=500
status_code=500, details=describe_exception(e)
)
@api_v3.route('/config/dim-schedule', methods=['POST'])
@@ -687,7 +690,7 @@ def save_dim_schedule_config():
ErrorCode.CONFIG_SAVE_FAILED,
"An error occurred; see logs for details",
status_code=500
status_code=500, details=describe_exception(e)
)
@api_v3.route('/config/main', methods=['POST'])
@@ -1314,7 +1317,7 @@ def save_main_config():
return error_response(
ErrorCode.CONFIG_SAVE_FAILED,
"An error occurred; see logs for details",
status_code=500
status_code=500, details=describe_exception(e)
)
@api_v3.route('/config/secrets', methods=['GET'])
@@ -1328,7 +1331,7 @@ def get_secrets_config():
return jsonify({'status': 'success', 'data': config})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/config/raw/main', methods=['POST'])
def save_raw_main_config():
@@ -1361,6 +1364,7 @@ def save_raw_main_config():
return error_response(
ErrorCode.CONFIG_SAVE_FAILED,
error_message,
details=describe_exception(e),
context={'config_path': e.config_path} if hasattr(e, 'config_path') and e.config_path else None,
status_code=500
@@ -1370,6 +1374,7 @@ def save_raw_main_config():
return error_response(
ErrorCode.UNKNOWN_ERROR,
error_message,
details=describe_exception(e),
status_code=500
)
@@ -1409,7 +1414,8 @@ def save_raw_secrets_config():
else:
error_message = 'An error occurred; see logs for details'
return jsonify({'status': 'error', 'message': error_message}), 500
return jsonify({'status': 'error', 'message': error_message,
'details': describe_exception(e)}), 500
@api_v3.route('/system/status', methods=['GET'])
def get_system_status():
@@ -1497,7 +1503,7 @@ def get_system_status():
return jsonify({'status': 'success', 'data': status})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/health', methods=['GET'])
def get_health():
@@ -1596,9 +1602,11 @@ def get_health():
return jsonify({'status': 'success', 'data': health_status})
except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details',
'details': describe_exception(e),
'data': {'status': 'unhealthy'}
}), 500
@@ -2368,7 +2376,7 @@ def get_display_current():
return jsonify({'status': 'success', 'data': display_data})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/display/on-demand/status', methods=['GET'])
def get_on_demand_status():
@@ -2392,7 +2400,7 @@ def get_on_demand_status():
})
except Exception as exc:
logger.error('Error in get_on_demand_status', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(exc)}), 500
@api_v3.route('/display/on-demand/start', methods=['POST'])
def start_on_demand_display():
@@ -2495,7 +2503,7 @@ def start_on_demand_display():
return jsonify({'status': 'success', 'data': response_data})
except Exception as exc:
logger.error('Error in start_on_demand_display', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(exc)}), 500
@api_v3.route('/display/on-demand/stop', methods=['POST'])
def stop_on_demand_display():
@@ -2531,7 +2539,7 @@ def stop_on_demand_display():
})
except Exception as exc:
logger.error('Error in stop_on_demand_display', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(exc)}), 500
@api_v3.route('/plugins/installed', methods=['GET'])
def get_installed_plugins():
@@ -2679,7 +2687,7 @@ def get_installed_plugins():
return jsonify({'status': 'success', 'data': {'plugins': plugins}})
except Exception as e:
logger.error('Error in get_installed_plugins', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
def _installed_plugin_ids():
"""Best-effort list of installed plugin IDs for the web process.
@@ -2745,7 +2753,7 @@ def get_plugin_health():
})
except Exception as e:
logger.error('Error in get_plugin_health', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/health/<plugin_id>', methods=['GET'])
def get_plugin_health_single(plugin_id):
@@ -2770,7 +2778,7 @@ def get_plugin_health_single(plugin_id):
})
except Exception as e:
logger.error('Error in get_plugin_health_single', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/health/<plugin_id>/reset', methods=['POST'])
def reset_plugin_health(plugin_id):
@@ -2795,7 +2803,7 @@ def reset_plugin_health(plugin_id):
})
except Exception as e:
logger.error('Error in reset_plugin_health', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/metrics', methods=['GET'])
def get_plugin_metrics():
@@ -2835,7 +2843,7 @@ def get_plugin_metrics():
})
except Exception as e:
logger.error('Error in get_plugin_metrics', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/metrics/<plugin_id>', methods=['GET'])
def get_plugin_metrics_single(plugin_id):
@@ -2860,7 +2868,7 @@ def get_plugin_metrics_single(plugin_id):
})
except Exception as e:
logger.error('Error in get_plugin_metrics_single', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/metrics/<plugin_id>/reset', methods=['POST'])
def reset_plugin_metrics(plugin_id):
@@ -2885,7 +2893,7 @@ def reset_plugin_metrics(plugin_id):
})
except Exception as e:
logger.error('Error in reset_plugin_metrics', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/limits/<plugin_id>', methods=['GET', 'POST'])
def manage_plugin_limits(plugin_id):
@@ -2940,7 +2948,7 @@ def manage_plugin_limits(plugin_id):
})
except Exception as e:
logger.error('Error in manage_plugin_limits', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/toggle', methods=['POST'])
def toggle_plugin():
@@ -3949,7 +3957,7 @@ def install_plugin():
except Exception as e:
logger.error('Error in install_plugin', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/install-from-url', methods=['POST'])
def install_plugin_from_url():
@@ -4004,7 +4012,7 @@ def install_plugin_from_url():
except Exception as e:
logger.error('Error in install_plugin_from_url', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/registry-from-url', methods=['POST'])
def get_registry_from_url():
@@ -4036,7 +4044,7 @@ def get_registry_from_url():
except Exception as e:
logger.error('Error in get_registry_from_url', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/saved-repositories', methods=['GET'])
def get_saved_repositories():
@@ -4049,7 +4057,7 @@ def get_saved_repositories():
return jsonify({'status': 'success', 'data': {'repositories': repositories}})
except Exception as e:
logger.error('Error in get_saved_repositories', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/saved-repositories', methods=['POST'])
def add_saved_repository():
@@ -4080,7 +4088,7 @@ def add_saved_repository():
}), 400
except Exception as e:
logger.error('Error in add_saved_repository', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/saved-repositories', methods=['DELETE'])
def remove_saved_repository():
@@ -4110,7 +4118,7 @@ def remove_saved_repository():
}), 404
except Exception as e:
logger.error('Error in remove_saved_repository', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/store/list', methods=['GET'])
def list_plugin_store():
@@ -4163,7 +4171,7 @@ def list_plugin_store():
return jsonify({'status': 'success', 'data': {'plugins': formatted_plugins}})
except Exception as e:
logger.error('Error in list_plugin_store', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/store/github-status', methods=['GET'])
def get_github_auth_status():
@@ -4214,7 +4222,7 @@ def get_github_auth_status():
})
except Exception as e:
logger.error('Error in get_github_auth_status', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/store/refresh', methods=['POST'])
def refresh_plugin_store():
@@ -4241,7 +4249,7 @@ def refresh_plugin_store():
})
except Exception as e:
logger.error('Error in refresh_plugin_store', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
def deep_merge(base_dict, update_dict):
"""
@@ -5763,7 +5771,7 @@ def get_plugin_schema():
return jsonify({'status': 'success', 'data': {'schema': default_schema}})
except Exception as e:
logger.error('Error in get_plugin_schema', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/skins', methods=['GET'])
def list_skins():
@@ -5798,9 +5806,9 @@ def list_skins():
'has_preview': bool(preview and (skin_dir / preview).is_file()),
})
return jsonify({'status': 'success', 'data': {'skins': payload}})
except Exception:
except Exception as e:
logger.error('Error in list_skins', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/config/reset', methods=['POST'])
def reset_plugin_config():
@@ -5880,7 +5888,7 @@ def reset_plugin_config():
})
except Exception as e:
logger.error('Error in reset_plugin_config', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/action', methods=['POST'])
def execute_plugin_action():
@@ -6140,7 +6148,7 @@ sys.exit(proc.returncode)
logger.error("Error executing action step 1", exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details'
'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500
else:
# Simple script execution
@@ -6190,7 +6198,7 @@ sys.exit(proc.returncode)
return jsonify({'status': 'error', 'message': 'Action timed out'}), 408
except Exception as e:
logger.error('Error in execute_plugin_action', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/authenticate/spotify', methods=['POST'])
def authenticate_spotify():
@@ -6323,12 +6331,12 @@ sys.exit(proc.returncode)
logger.error("Error getting Spotify auth URL", exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details'
'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500
except Exception as e:
logger.error('Error in authenticate_spotify', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/authenticate/ytm', methods=['POST'])
def authenticate_ytm():
@@ -6378,7 +6386,7 @@ def authenticate_ytm():
return jsonify({'status': 'error', 'message': 'Authentication timed out'}), 408
except Exception as e:
logger.error('Error in authenticate_ytm', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/catalog', methods=['GET'])
def get_fonts_catalog():
@@ -6473,7 +6481,10 @@ def get_fonts_catalog():
return jsonify({'status': 'success', 'data': {'catalog': catalog}})
except Exception as e:
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
logger.error("%s failed", request.path, exc_info=True)
return jsonify({'status': 'error',
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)}), 500
@api_v3.route('/fonts/tokens', methods=['GET'])
def get_font_tokens():
@@ -6492,7 +6503,7 @@ def get_font_tokens():
return jsonify({'status': 'success', 'data': {'tokens': tokens}})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/overrides', methods=['GET'])
def get_fonts_overrides():
@@ -6504,7 +6515,7 @@ def get_fonts_overrides():
return jsonify({'status': 'success', 'data': {'overrides': overrides}})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/overrides', methods=['POST'])
def save_fonts_overrides():
@@ -6518,7 +6529,7 @@ def save_fonts_overrides():
return jsonify({'status': 'success', 'message': 'Font overrides saved'})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/overrides/<element_key>', methods=['DELETE'])
def delete_font_override(element_key):
@@ -6528,7 +6539,7 @@ def delete_font_override(element_key):
return jsonify({'status': 'success', 'message': f'Font override for {element_key} deleted'})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/upload', methods=['POST'])
def upload_font():
@@ -6593,7 +6604,7 @@ def upload_font():
})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/preview', methods=['GET'])
@@ -6738,7 +6749,7 @@ def get_font_preview() -> tuple[Response, int] | Response:
})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/fonts/<font_family>', methods=['DELETE'])
@@ -6826,7 +6837,7 @@ def delete_font(font_family: str) -> tuple[Response, int] | Response:
})
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/assets/upload', methods=['POST'])
@@ -6974,7 +6985,7 @@ def upload_plugin_asset():
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/of-the-day/json/upload', methods=['POST'])
def upload_of_the_day_json():
@@ -7124,7 +7135,7 @@ def upload_of_the_day_json():
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/of-the-day/json/delete', methods=['POST'])
def delete_of_the_day_json():
@@ -7171,7 +7182,7 @@ def delete_of_the_day_json():
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/<plugin_id>/static/<path:file_path>', methods=['GET'])
def serve_plugin_static(plugin_id, file_path):
@@ -7217,7 +7228,7 @@ def serve_plugin_static(plugin_id, file_path):
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/calendar/upload-credentials', methods=['POST'])
@@ -7299,7 +7310,223 @@ def upload_calendar_credentials():
except Exception as e:
logger.error('Error in upload_calendar_credentials', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
# calendarList.list pages at 250 entries maximum. Ten pages is far past any
# real account and exists only so a malformed nextPageToken cannot spin here.
_CALENDAR_LIST_MAX_PAGES = 10
def _calendar_plugin_dir() -> Optional[Path]:
"""Where the calendar plugin is installed, or None if it is not."""
if api_v3.plugin_manager:
plugin_dir = api_v3.plugin_manager.get_plugin_directory('calendar')
else:
plugin_dir = PROJECT_ROOT / 'plugins' / 'calendar'
if not plugin_dir:
return None
plugin_dir = Path(plugin_dir)
return plugin_dir if plugin_dir.exists() else None
def _run_calendar_registration(plugin_dir: Path, stdin_payload: str):
"""Run the plugin's OAuth script and return the JSON object it prints.
The script decides between web and terminal mode by whether stdin is a
tty, so it must be given a pipe. It emits one JSON object on stdout; the
last parsable line is taken, because an import warning or a library's
stderr redirection can land in front of it.
Returns (payload, error_message). Exactly one is None.
"""
script = plugin_dir / 'calendar_registration.py'
if not script.exists():
return None, 'Authentication script not found in the calendar plugin'
try:
result = subprocess.run( # nosec B603 - fixed script path inside the plugin dir
[sys.executable, str(script)],
input=stdin_payload,
capture_output=True,
text=True,
timeout=120,
cwd=str(plugin_dir),
)
except subprocess.TimeoutExpired:
return None, 'Authentication timed out after 120s'
except OSError as e:
return None, 'Could not run the authentication script: %s' % e
for line in reversed((result.stdout or '').splitlines()):
line = line.strip()
if not line:
continue
try:
payload = json.loads(line)
except json.JSONDecodeError:
continue
if isinstance(payload, dict):
return payload, None
raw = (result.stderr or result.stdout or '').strip()
# The unredacted text goes to the log, where it is worth having in full.
# What comes back over HTTP is redacted: this is a script that handles
# OAuth client secrets, and its stderr can quote them.
if raw:
logger.error('calendar_registration.py failed (exit %s): %s',
result.returncode, raw)
return None, 'Authentication script produced no result%s' % (
': %s' % redact_text(raw) if raw else '')
@api_v3.route('/plugins/calendar/authenticate', methods=['POST'])
def authenticate_calendar():
"""Google OAuth for the calendar plugin, in the two steps it requires.
Step 1 (no body) returns the consent URL to open. Step 2 posts back the
URL Google redirected to -- it fails to load, because the redirect points
at a loopback address nothing is listening on, but the address bar carries
the authorization code -- and the script exchanges it for a token.
Two calls rather than one because the user has to visit Google in between.
The script persists the PKCE verifier from step 1 for step 2 to reuse; the
exchange fails with "Missing code verifier" otherwise.
"""
try:
plugin_dir = _calendar_plugin_dir()
if plugin_dir is None:
return jsonify({
'status': 'error',
'message': 'The calendar plugin is not installed'
}), 404
if not (plugin_dir / 'credentials.json').exists():
return jsonify({
'status': 'error',
'message': ('No credentials.json yet. Upload your Google OAuth '
'client file first (Step 1).')
}), 400
data = request.get_json(silent=True) or {}
redirect_url = (data.get('redirect_url') or data.get('code') or '').strip()
payload, error = _run_calendar_registration(plugin_dir, redirect_url)
if error:
return jsonify({'status': 'error', 'message': error}), 500
if payload.get('status') != 'success':
# The script's own diagnosis is more useful than anything that
# could be reconstructed here -- but it interpolates exceptions
# into its messages, so it reaches the client redacted and the
# original goes to the log.
logger.error('calendar authentication failed: %s', payload)
safe = dict(payload)
safe['message'] = redact_text(str(payload.get('message', '')
or 'Authentication failed'))
return jsonify(safe), 400
return jsonify(payload)
except Exception as e:
logger.error('Error in authenticate_calendar', exc_info=True)
return jsonify({'status': 'error',
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)}), 500
@api_v3.route('/plugins/calendar/list-calendars', methods=['GET'])
def list_calendar_calendars():
"""The calendars this account can see, for the config picker.
Reads the token the OAuth flow wrote rather than shelling out again: the
picker is used interactively and a subprocess per click is slower than the
API call it would be wrapping.
"""
try:
plugin_dir = _calendar_plugin_dir()
if plugin_dir is None:
return jsonify({
'status': 'error',
'message': 'The calendar plugin is not installed'
}), 404
token_file = plugin_dir / 'token.pickle'
if not token_file.exists():
return jsonify({
'status': 'error',
'message': ('Not authenticated with Google yet. Complete Step 2 '
'first, then load your calendars.')
}), 400
try:
import pickle
from google.auth.transport.requests import Request as GoogleRequest
from googleapiclient.discovery import build as build_google_service
except ImportError as e:
return jsonify({
'status': 'error',
'message': ('The Google API libraries are not installed. Install '
"the calendar plugin's requirements.txt. (%s)" % e)
}), 500
with open(token_file, 'rb') as handle:
# Written only by this plugin's own OAuth flow, into its own
# directory, and read here exactly as the plugin itself reads it.
creds = pickle.load(handle) # nosec B301 - locally generated token
if creds and creds.expired and creds.refresh_token:
creds.refresh(GoogleRequest())
with open(token_file, 'wb') as handle:
pickle.dump(creds, handle)
os.chmod(token_file, 0o600)
if not creds or not creds.valid:
return jsonify({
'status': 'error',
'message': ('Stored Google credentials are no longer valid. '
'Run Step 2 again to re-authenticate.')
}), 400
service = build_google_service('calendar', 'v3', credentials=creds)
# calendarList.list returns 100 entries per page by default and caps at
# 250, handing back a nextPageToken when there are more. Taking only
# the first page would silently hide calendars from the picker, and the
# user would have no way to tell the list was truncated.
entries = []
page_token = None
for _ in range(_CALENDAR_LIST_MAX_PAGES):
response = service.calendarList().list(
maxResults=250, pageToken=page_token).execute()
entries.extend(response.get('items', []))
page_token = response.get('nextPageToken')
if not page_token:
break
else:
# 2500 calendars in, something is wrong with the account or the
# token is looping; show what was collected rather than spin.
logger.warning(
'calendarList paging stopped at %d pages with more remaining',
_CALENDAR_LIST_MAX_PAGES)
calendars = [{
'id': entry.get('id'),
# The picker labels each row with summary and falls back to the id
# only in its own display, so send something either way.
'summary': entry.get('summary') or entry.get('id'),
'primary': bool(entry.get('primary', False)),
} for entry in entries if entry.get('id')]
# Primary first, then alphabetically: the list is usually short but the
# one the user wants is almost always their own calendar.
calendars.sort(key=lambda c: (not c['primary'], c['summary'].lower()))
return jsonify({'status': 'success', 'calendars': calendars})
except Exception as e:
logger.error('Error in list_calendar_calendars', exc_info=True)
return jsonify({'status': 'error',
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)}), 500
@api_v3.route('/plugins/assets/delete', methods=['POST'])
def delete_plugin_asset():
@@ -7342,7 +7569,7 @@ def delete_plugin_asset():
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/plugins/assets/list', methods=['GET'])
def list_plugin_assets():
@@ -7370,7 +7597,7 @@ def list_plugin_assets():
except Exception as e:
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/display/current-status', methods=['GET'])
def get_current_display_status():
@@ -7391,9 +7618,9 @@ def get_current_display_status():
'last_updated': None,
}
return jsonify({'status': 'success', 'data': state})
except Exception:
except Exception as e:
logger.error('Error in get_current_display_status', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/logs', methods=['GET'])
def get_logs():
@@ -7432,9 +7659,11 @@ def get_logs():
'message': 'Timeout while fetching logs'
}), 500
except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details'
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)
}), 500
# Multi-Display Sync Endpoints
@@ -7499,9 +7728,11 @@ def get_wifi_status():
}
})
except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details'
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)
}), 500
@api_v3.route('/wifi/scan', methods=['GET'])
@@ -7623,7 +7854,7 @@ def connect_wifi():
logger.error("Error connecting to WiFi", exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details'
'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500
@api_v3.route('/wifi/disconnect', methods=['POST'])
@@ -7649,7 +7880,7 @@ def disconnect_wifi():
logger.error("Error disconnecting from WiFi", exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details'
'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500
@api_v3.route('/wifi/ap/enable', methods=['POST'])
@@ -7674,9 +7905,11 @@ def enable_ap_mode():
'message': message
}), 400
except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details'
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)
}), 500
@api_v3.route('/wifi/ap/disable', methods=['POST'])
@@ -7699,9 +7932,11 @@ def disable_ap_mode():
'message': message
}), 400
except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details'
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)
}), 500
@api_v3.route('/wifi/ap/auto-enable', methods=['GET'])
@@ -7720,9 +7955,11 @@ def get_auto_enable_ap_mode():
}
})
except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details'
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)
}), 500
@api_v3.route('/wifi/ap/auto-enable', methods=['POST'])
@@ -7752,9 +7989,11 @@ def set_auto_enable_ap_mode():
}
})
except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details'
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)
}), 500
@api_v3.route('/wifi/radio', methods=['GET'])
@@ -7774,7 +8013,7 @@ def get_wifi_radio():
logger.error("Error getting WiFi radio state", exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details'
'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500
@api_v3.route('/wifi/radio', methods=['POST'])
@@ -7822,7 +8061,7 @@ def set_wifi_radio():
logger.error("Error setting WiFi radio state", exc_info=True)
return jsonify({
'status': 'error',
'message': 'An error occurred; see logs for details'
'message': 'An error occurred; see logs for details', 'details': describe_exception(e)
}), 500
@api_v3.route('/cache/list', methods=['GET'])
@@ -7847,7 +8086,7 @@ def list_cache_files():
})
except Exception as e:
logger.error('Error in list_cache_files', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
@api_v3.route('/cache/delete', methods=['POST'])
def delete_cache_file():
@@ -7873,7 +8112,7 @@ def delete_cache_file():
})
except Exception as e:
logger.error('Error in delete_cache_file', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500
# =============================================================================
@@ -0,0 +1,184 @@
/**
* Google OAuth Widget
*
* Step 2 of the calendar plugin's setup, between uploading the OAuth client
* file and picking calendars. Google will not let a headless device complete
* consent on its own, so the flow is necessarily two calls with a human in
* between:
*
* 1. POST /api/v3/plugins/calendar/authenticate with no body
* -> { auth_url } to open in a browser
* 2. the browser lands on a loopback address that fails to load; its URL
* carries the authorization code. POST it back as redirect_url
* -> the server exchanges it and writes token.pickle
*
* The failed page in step 2 is expected and is worth saying out loud, because
* it looks exactly like something went wrong.
*
* @module GoogleOAuthWidget
*/
(function () {
'use strict';
if (typeof window.LEDMatrixWidgets === 'undefined') {
console.error('[GoogleOAuthWidget] LEDMatrixWidgets registry not found. Load registry.js first.');
return;
}
const ENDPOINT = '/api/v3/plugins/calendar/authenticate';
window.LEDMatrixWidgets.register('google-oauth', {
name: 'Google OAuth Widget',
version: '1.0.0',
/**
* @param {HTMLElement} container
* @param {Object} config - schema config (unused)
* @param {*} value - unused; this widget stores nothing
* @param {Object} options - { fieldId, pluginId, name }
*/
render: function (container, config, value, options) {
const fieldId = options.fieldId;
// Nothing is stored in config by this step -- the result is
// token.pickle on the device -- but the form still expects a field.
const hidden = document.createElement('input');
hidden.type = 'hidden';
hidden.id = fieldId + '_hidden';
hidden.name = options.name;
hidden.value = value || '';
const startBtn = document.createElement('button');
startBtn.type = 'button';
startBtn.className = 'px-3 py-1.5 text-sm rounded-md bg-blue-600 hover:bg-blue-700 text-white';
startBtn.innerHTML = '<i class="fas fa-key"></i> Connect Google Account';
const status = document.createElement('p');
status.className = 'text-xs text-gray-400 mt-2';
const step2 = document.createElement('div');
step2.className = 'mt-3 hidden';
const link = document.createElement('a');
link.target = '_blank';
link.rel = 'noopener noreferrer';
link.className = 'text-blue-400 underline text-sm break-all';
link.textContent = 'Open the Google consent screen';
// Deliberately loud. After consent the browser is redirected to a
// loopback address nothing is listening on, so it lands on a
// browser error page -- which reads as a failure at exactly the
// moment the user has to act on it. Said quietly in grey it gets
// missed, and the flow looks broken when it is working.
const hint = document.createElement('div');
hint.className =
'mt-3 p-3 rounded-md border border-amber-500/60 bg-amber-500/10';
hint.innerHTML =
'<p class="text-sm text-amber-300 font-semibold">'
+ '<i class="fas fa-triangle-exclamation"></i> '
+ 'The next page will fail to load. That is expected.</p>'
+ '<p class="text-xs text-amber-200/90 mt-1">'
+ 'After you approve access, Google sends your browser to '
+ '<code>127.0.0.1</code>, where nothing is running \u2014 so you will see '
+ '"This site can\u2019t be reached" or similar. Nothing has gone wrong. '
+ 'Copy the <strong>entire address</strong> out of the address bar '
+ '(it contains <code>?code=...</code>) and paste it in the box below.</p>';
const codeLabel = document.createElement('label');
codeLabel.className = 'block text-xs text-gray-300 mt-3';
codeLabel.textContent = 'Paste the address from that failed page here:';
const codeInput = document.createElement('input');
codeInput.type = 'text';
codeInput.placeholder = 'http://127.0.0.1/?code=...';
codeInput.className =
'mt-2 block w-full px-3 py-2 text-sm border border-gray-600 '
+ 'rounded-md bg-gray-800 text-gray-100';
const finishBtn = document.createElement('button');
finishBtn.type = 'button';
finishBtn.className = 'mt-2 px-3 py-1.5 text-sm rounded-md bg-green-600 hover:bg-green-700 text-white';
finishBtn.innerHTML = '<i class="fas fa-check"></i> Finish Authentication';
function say(message, kind) {
status.textContent = message;
status.className = 'text-xs mt-2 ' + (
kind === 'error' ? 'text-red-400'
: kind === 'success' ? 'text-green-400'
: 'text-gray-400');
}
function post(body) {
return fetch(ENDPOINT, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body || {})
}).then(function (r) {
return r.json().catch(function () {
// A non-JSON body here means the request never reached
// the handler -- worth saying so rather than "undefined".
return { status: 'error', message: 'Server returned ' + r.status };
});
});
}
startBtn.addEventListener('click', function () {
startBtn.disabled = true;
say('Requesting a consent link...');
post({}).then(function (data) {
startBtn.disabled = false;
if (data.status !== 'success' || !data.auth_url) {
say(data.message || 'Could not start authentication.', 'error');
return;
}
link.href = data.auth_url;
step2.classList.remove('hidden');
say(data.message || 'Open the link, approve, then paste the address back.');
}).catch(function (err) {
startBtn.disabled = false;
say('Request failed: ' + err.message, 'error');
});
});
finishBtn.addEventListener('click', function () {
const pasted = codeInput.value.trim();
if (!pasted) {
say('Paste the address your browser was redirected to.', 'error');
return;
}
finishBtn.disabled = true;
say('Exchanging the code with Google...');
post({ redirect_url: pasted }).then(function (data) {
finishBtn.disabled = false;
if (data.status !== 'success') {
say(data.message || 'Authentication failed.', 'error');
return;
}
say(data.message || 'Authenticated.', 'success');
step2.classList.add('hidden');
codeInput.value = '';
}).catch(function (err) {
finishBtn.disabled = false;
say('Request failed: ' + err.message, 'error');
});
});
step2.appendChild(link);
step2.appendChild(hint);
step2.appendChild(codeLabel);
step2.appendChild(codeInput);
step2.appendChild(finishBtn);
container.appendChild(hidden);
container.appendChild(startBtn);
container.appendChild(status);
container.appendChild(step2);
},
getValue: function (fieldId) {
const hidden = document.getElementById(fieldId + '_hidden');
return hidden ? hidden.value : '';
}
});
})();
+1
View File
@@ -987,6 +987,7 @@
<script src="{{ url_for('static', filename='v3/js/widgets/custom-feeds.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/array-table.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/google-calendar-picker.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/google-oauth.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/day-selector.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/time-range.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/time-picker.js') }}" defer></script>
@@ -815,7 +815,7 @@
<i class="fas fa-info-circle mr-1"></i>
Changes in the file manager save immediately — no need to click Save Configuration.
</p>
{% elif str_widget in ['text-input', 'textarea', 'select-dropdown', 'toggle-switch', 'radio-group', 'date-picker', 'time-picker', 'slider', 'color-picker', 'email-input', 'url-input', 'password-input', 'font-selector', 'file-upload-single', 'plugin-file-manager'] %}
{% elif str_widget in ['text-input', 'textarea', 'select-dropdown', 'toggle-switch', 'radio-group', 'date-picker', 'time-picker', 'slider', 'color-picker', 'email-input', 'url-input', 'password-input', 'font-selector', 'file-upload-single', 'plugin-file-manager', 'google-oauth'] %}
{# Render widget container #}
<div id="{{ field_id }}_container" class="{{ str_widget }}-container"></div>
<script>