Compare commits

..
Author SHA1 Message Date
ChuckBuildsandClaude Opus 5 ecf9195c11 fix(odds): stop a stalled ESPN taking the whole plugin update with it
Odds are fetched per live game from inside SportsLive.update(), with
show_odds defaulting on, and the plugin executor kills an operation at
30s. The odds request timeout was also 30s, so a single stalled request
consumed the entire budget and the update carrying every game's score
was killed.

Out of season that is invisible: preseason week 1 returns one game. A
Sunday slate is around sixteen, so the odds of at least one slow request
rise sharply just as the cost of losing the update does.

Shorten the request timeout to 5s, and after a network failure skip the
network for 60s. The timeout alone is not enough -- sixteen consecutive
5s timeouts still blow through -- and when ESPN is unreachable it is
unreachable for the whole slate, so the first failure already answers
the question for the rest of the pass.

    before: one stalled request = 30s = the entire budget
    after : 5s, the rest of the slate skipped, retry after 60s

The stale-cache fallback is unchanged: the cache is consulted before any
of this, and the failing request still falls back to it.

An earlier version of this branch also jittered the cache TTL to stagger
expiry across a slate. That has been dropped: CacheManager.set() stores
ttl for compatibility but the read path expires entries by a per-type
max_age (1800s for odds), so the jitter was inert. Making the read path
honour a per-entry ttl is a real fix but changes a contract 48 plugin
call sites already rely on, which is not a change to make two weeks
before the season.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5
2026-08-11 11:32:56 -04:00
37 changed files with 160 additions and 3800 deletions
-8
View File
@@ -600,14 +600,6 @@ These settings are typically only needed for non-standard panels or custom confi
- Leave empty unless you need custom mapping - Leave empty unless you need custom mapping
- See rpi-rgb-led-matrix documentation for full options - See rpi-rgb-led-matrix documentation for full options
- **`orientation`** (string, default: "normal")
- Rotates the rendered image to match how the panel is physically mounted
- Set to `"180"` (or use the "Upside Down" option in the web UI's Display
settings) if the panel is mounted upside down — useful for optimizing
where the Raspberry Pi and wiring sit relative to the mounting location
- Applied independently of `pixel_mapper_config` (appended as a trailing
`Rotate:180` mapper), so custom mapper configs keep working alongside it
- **`row_address_type`** (integer, default: 0) - **`row_address_type`** (integer, default: 0)
- How rows are addressed on the panel - How rows are addressed on the panel
- Most panels use 0 (direct addressing) - Most panels use 0 (direct addressing)
-4
View File
@@ -112,7 +112,6 @@
"led_rgb_sequence": "RGB", "led_rgb_sequence": "RGB",
"limit_refresh_rate_hz": 100, "limit_refresh_rate_hz": 100,
"pixel_mapper_config": "", "pixel_mapper_config": "",
"orientation": "normal",
"row_address_type": 0, "row_address_type": 0,
"multiplexing": 0, "multiplexing": 0,
"panel_type": "" "panel_type": ""
@@ -130,9 +129,6 @@
"plugin_rotation_order": [], "plugin_rotation_order": [],
"use_short_date_format": true, "use_short_date_format": true,
"vegas_scroll": { "vegas_scroll": {
"live_in_ticker": false,
"live_weight": 3,
"favorite_live_weight": 5,
"enabled": false, "enabled": false,
"scroll_speed": 50, "scroll_speed": 50,
"separator_width": 32, "separator_width": 32,
+1 -89
View File
@@ -64,98 +64,10 @@ JSON is optional.
| `target_fps` | `125` | Target frame rate | | `target_fps` | `125` | Target frame rate |
| `buffer_ahead` | `2` | Number of plugins buffered ahead | | `buffer_ahead` | `2` | Number of plugins buffered ahead |
This table is a subset — `display.vegas_scroll` supports 30 keys in This table is a subset — `display.vegas_scroll` supports 26 keys in
total. See the full list in total. See the full list in
[CONFIG_REFERENCE.md](CONFIG_REFERENCE.md#displayvegas_scroll--continuous-scroll-mode). [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md#displayvegas_scroll--continuous-scroll-mode).
### Live Content in the Ticker
By default, live content **preempts** Vegas mode: while any plugin reports
live priority, the display controller refuses to run the ticker and shows
that plugin's full-screen display instead. You get a big readable scoreboard,
but the marquee stops entirely for the duration of the game.
Set `live_in_ticker` to keep the ticker running and let live content take
**extra turns inside it** instead:
```json
"vegas_scroll": {
"live_in_ticker": true,
"live_weight": 3,
"favorite_live_weight": 5
}
```
#### Why weights exist
The rotation is otherwise a strict round robin — every plugin appears exactly
once per cycle. With a dozen plugins enabled, a live score comes round once a
lap and can be minutes old by the time you see it. A weight of *N* gives a
plugin *N* slots per cycle.
The slots are placed by **Smooth Weighted Round-Robin**, the same scheduler
the sports plugins use internally to rotate their own games. The important
property is that repeats are *spread through the cycle* rather than clumped:
three appearances in a row followed by a long silence would be worse than not
boosting at all.
Twelve plugins, with a favorite's baseball game and an ordinary live hockey
game (`live_weight: 3`, `favorite_live_weight: 5`):
```
baseball > hockey > weather > clock > baseball
stocks > news > flights > baseball > hockey
calendar > f1 > music > baseball > tides
birds > hockey > baseball
```
18 slots for 12 plugins. Baseball appears 5 times, hockey 3, everything else
once, and no plugin ever appears twice in a row — **including across the seam**
where the cycle loops back on itself. Smooth Weighted Round-Robin schedules the
heaviest item first and usually last as well, so the strip would otherwise show
it twice running at exactly the one join a within-cycle check cannot see. The
trailing repeat is moved into the widest remaining gap. Where a double is
unavoidable — a plugin holding most of the slots has to neighbour itself — the
schedule is left as it is.
#### Where the weight comes from
For each plugin in the rotation, in order:
1. **The plugin's own answer.** If it implements
`get_vegas_priority_weight()` and returns a number, that wins. This is the
only route for favorite-team awareness — the core can see *that* a game is
live, but not *whose*, so a scoreboard has to say so itself.
2. **The core's default.** When the plugin returns `None` (the base-class
default), a plugin where both `has_live_priority()` and `has_live_content()`
are true gets `live_weight`.
3. **Everything else** gets 1.
Because of step 2, **existing plugins need no changes** — any scoreboard with
`live_priority` enabled already gets extra turns. Step 1 is opt-in, for
plugins that want to distinguish a favorite's game from any other live game.
Weights are clamped to 110. A weight of 1 is no boost; a weight below 1 would
drop the plugin from the rotation entirely, which is never what is meant.
#### Things worth knowing
- **Weights are per plugin, not per game.** A scoreboard showing four live
games still occupies one slot at a time, rotating its own games within that
slot using its own `favorite_live_boost`. This controls how often the
*plugin* comes round.
- **The ticker is zero-sum.** Giving baseball 5 slots does not make the cycle
faster; it makes the cycle *longer* and everything else proportionally
rarer. If you want live scores sooner in wall-clock terms, pair this with a
smaller `plugins_per_cycle`.
- **Frequency is not freshness.** Each appearance redraws from the plugin's
current data (`refresh_updated_plugins()` drops cached content when a
plugin's data changes), but how current that data is depends on the
plugin's own `live_update_interval`. Showing a stale score five times a lap
is no better than showing it once.
- **Everything still appears.** A boost never starves another plugin out of
the cycle; low-weight plugins keep their single slot.
### Per-Plugin Configuration ### Per-Plugin Configuration
Override Vegas behavior for specific plugins: Override Vegas behavior for specific plugins:
+1 -6
View File
@@ -66,7 +66,6 @@ in `DisplayManager` (`src/display_manager.py`, ~lines 270295).
| `led_rgb_sequence` | string, `"RGB"` | | `led_rgb_sequence` | string, `"RGB"` |
| `limit_refresh_rate_hz` | int, `100` (code default 90) | | `limit_refresh_rate_hz` | int, `100` (code default 90) |
| `pixel_mapper_config` | string, `""` — e.g. `"U-mapper"` / `"Rotate:90"` | | `pixel_mapper_config` | string, `""` — e.g. `"U-mapper"` / `"Rotate:90"` |
| `orientation` | string, `"normal"``"180"` rotates the rendered image 180° for panels physically mounted upside down (e.g. to move the Pi/wiring to a more convenient side); composed onto `pixel_mapper_config` as a trailing `Rotate:180` mapper, so it stays independent of any custom `pixel_mapper_config` value |
| `row_address_type` | int, `0` — non-standard panel row addressing | | `row_address_type` | int, `0` — non-standard panel row addressing |
| `multiplexing` | int, `0` — panel multiplexing scheme | | `multiplexing` | int, `0` — panel multiplexing scheme |
| `panel_type` | string, `""` — set to `"FM6126A"` or `"FM6127"` for panels needing init | | `panel_type` | string, `""` — set to `"FM6126A"` or `"FM6127"` for panels needing init |
@@ -104,8 +103,7 @@ logical image to multiple chained physical panels.
## `display.vegas_scroll` — continuous scroll mode ## `display.vegas_scroll` — continuous scroll mode
Read by `src/vegas_mode/config.py` (`VegasScrollConfig.from_config`). See Read by `src/vegas_mode/config.py` (`VegasScrollConfig.from_config`). See
[ADVANCED_FEATURES.md](ADVANCED_FEATURES.md) for behavior details, including [ADVANCED_FEATURES.md](ADVANCED_FEATURES.md) for behavior details.
[live content in the ticker](ADVANCED_FEATURES.md#live-content-in-the-ticker).
| Key | Type / default | | Key | Type / default |
|---|---| |---|---|
@@ -136,9 +134,6 @@ Read by `src/vegas_mode/config.py` (`VegasScrollConfig.from_config`). See
| `max_cycle_duration` | int, `240` | | `max_cycle_duration` | int, `240` |
| `frame_based_scrolling` | bool, `true` — frame-count-based scroll stepping | | `frame_based_scrolling` | bool, `true` — frame-count-based scroll stepping |
| `scroll_delay` | float, `0.02` — seconds between scroll updates (~50 FPS) | | `scroll_delay` | float, `0.02` — seconds between scroll updates (~50 FPS) |
| `live_in_ticker` | bool, `false` — keep scrolling during live games instead of handing the display to a full-screen scoreboard |
| `live_weight` | int, `3` (110) — slots per cycle for a plugin with live content |
| `favorite_live_weight` | int, `5` (110) — slots per cycle when a plugin reports a favorite team is live |
## `sync` — multi-display synchronization ## `sync` — multi-display synchronization
-41
View File
@@ -170,47 +170,6 @@ Default returns `False`.
List of display modes to show during a live takeover. Default returns the List of display modes to show during a live takeover. Default returns the
plugin's `display_modes` from its manifest. plugin's `display_modes` from its manifest.
#### `get_vegas_priority_weight() -> Optional[int]`
How many slots per Vegas cycle this plugin should get. Default returns
`None`, which defers to the core.
The Vegas ticker is otherwise a strict round robin — every plugin appears
exactly once per cycle — so with a dozen plugins enabled a live score can be
minutes stale by the time it comes round. A weight of *N* gives the plugin
*N* slots per cycle, spread evenly through it rather than clumped.
**You usually do not need this.** When the hook returns `None`, the core
already gives a plugin `vegas_scroll.live_weight` whenever
`has_live_priority()` and `has_live_content()` are both true. Live sports get
extra turns with no code at all.
Implement it only when the plugin knows something the core cannot. The
motivating case is favorite teams — the core can see *that* a game is live,
but not *whose*:
```python
def get_vegas_priority_weight(self):
if not (self.has_live_priority() and self.has_live_content()):
return None # let the core decide
vegas = self.global_config.get('display', {}).get('vegas_scroll', {})
if self._favorite_is_live():
return vegas.get('favorite_live_weight', 5)
return vegas.get('live_weight', 3)
```
The weight is per *plugin*, not per game: a scoreboard showing four live games
still occupies one slot at a time and rotates its own games within it. Values
are clamped to 110 by the caller. An exception here is caught and logged, and
the core then falls back to its own live-content check — so a plugin whose
weight calculation is broken still gets `live_weight` for a game that really
is live, rather than being demoted to 1.
Only consulted when the user has set `vegas_scroll.live_in_ticker`. With the
default (`false`) live content preempts Vegas entirely and there is no ticker
to be weighted within. See
[ADVANCED_FEATURES.md](ADVANCED_FEATURES.md#live-content-in-the-ticker).
### Vegas scroll hooks ### Vegas scroll hooks
Vegas mode shows multiple plugins as a single continuous scroll instead of Vegas mode shows multiple plugins as a single continuous scroll instead of
+1 -19
View File
@@ -45,24 +45,6 @@ class BaseOddsManager:
self.logger = logging.getLogger(__name__) self.logger = logging.getLogger(__name__)
self.base_url = "https://sports.core.api.espn.com/v2/sports" self.base_url = "https://sports.core.api.espn.com/v2/sports"
# This path used a bare requests.get, so it identified itself as
# python-requests/x.y -- the one thing ESPN is known to reject. Around
# 2026-08-04 it began 403ing browser strings and bare custom tokens
# alike; what it accepts is a token with a URL that says who is
# calling. Every other ESPN caller in the tree already sends this
# (src/common/api_helper.py, src/base_classes/data_sources.py); the
# odds path was simply missed, and it is the one whose failures cost
# the caller its whole update budget.
#
# Deliberately no retry adapter, unlike api_helper: retries multiply
# request_timeout, which is set to 5s precisely to stay inside that
# budget. One try, then the cooldown below.
self.session = requests.Session()
self.session.headers.update({
'User-Agent': 'LEDMatrix/1.0 (+https://github.com/ChuckBuilds/LEDMatrix)',
'Accept': 'application/json',
})
# Configuration with defaults # Configuration with defaults
self.update_interval = 3600 # 1 hour default self.update_interval = 3600 # 1 hour default
# Well under the plugin executor's 30s operation budget. At 30s a # Well under the plugin executor's 30s operation budget. At 30s a
@@ -162,7 +144,7 @@ class BaseOddsManager:
url = f"{self.base_url}/{sport}/leagues/{espn_league}/events/{event_id}/competitions/{event_id}/odds" url = f"{self.base_url}/{sport}/leagues/{espn_league}/events/{event_id}/competitions/{event_id}/odds"
self.logger.info(f"Requesting odds from URL: {url}") self.logger.info(f"Requesting odds from URL: {url}")
response = self.session.get(url, timeout=self.request_timeout) response = requests.get(url, timeout=self.request_timeout)
response.raise_for_status() response.raise_for_status()
raw_data = response.json() raw_data = response.json()
+1 -80
View File
@@ -14,13 +14,6 @@ import zlib
from typing import Dict, Any, Optional, Protocol from typing import Dict, Any, Optional, Protocol
from datetime import datetime from datetime import datetime
# How old an abandoned write's temp file must be before the sweep removes it.
# A real write holds its temp file for milliseconds, so an hour is far beyond
# any in-flight write while still clearing the same day's debris. Deliberately
# not tied to the retention policies: those describe how long data stays
# useful, and a half-written file was never useful.
_ORPHAN_TEMP_MAX_AGE_SECONDS = 3600
class CacheStrategyProtocol(Protocol): class CacheStrategyProtocol(Protocol):
@@ -119,22 +112,6 @@ class DiskCache:
record_ts = None record_ts = None
now = time.time() now = time.time()
# An explicit per-entry ttl wins over the caller's max_age. The
# caller that wrote the record knows what its data is; max_age is
# inferred from substrings in the key ("live", "odds", "stock") and
# is only a fallback for records that never said. Until now the ttl
# was stored and ignored, so `set(key, data, ttl=...)` did nothing
# at all -- 48 plugin call sites and 4 in the core were writing a
# number no read path consulted.
effective_max_age = max_age
if isinstance(record, dict):
stored_ttl = record.get('ttl')
if isinstance(stored_ttl, (int, float)) and not isinstance(stored_ttl, bool) \
and stored_ttl >= 0:
effective_max_age = stored_ttl
max_age = effective_max_age
# max_age=None means "never expires" (mirrors MemoryCache and the # max_age=None means "never expires" (mirrors MemoryCache and the
# cache_manager docstring). Guard it explicitly — otherwise the # cache_manager docstring). Guard it explicitly — otherwise the
# comparison below raises TypeError and the record is treated as a # comparison below raises TypeError and the record is treated as a
@@ -354,23 +331,6 @@ class DiskCache:
"""Get the cache directory path.""" """Get the cache directory path."""
return self.cache_dir return self.cache_dir
@staticmethod
def _is_orphaned_temp(filename: str) -> bool:
"""Whether a name is one of set()'s temp files rather than real data.
Matches only what this class creates: mkstemp with a prefix of
".<cache filename>." , so ".weather.json.a1b2c3d4". The shape is
checked rather than just the leading dot, because this predicate
deletes things -- a stray dotfile someone left in the cache directory
is not ours to remove, and a completed ".json" never is either.
"""
if not filename.startswith('.') or filename.endswith('.json'):
return False
head, sep, suffix = filename.rpartition('.json.')
# head is the key (non-empty after the leading dot), suffix is
# mkstemp's random component.
return bool(sep) and len(head) > 1 and bool(suffix)
def cleanup_expired_files(self, cache_strategy: CacheStrategyProtocol, retention_policies: Dict[str, int]) -> Dict[str, Any]: def cleanup_expired_files(self, cache_strategy: CacheStrategyProtocol, retention_policies: Dict[str, int]) -> Dict[str, Any]:
""" """
Clean up expired cache files based on retention policies. Clean up expired cache files based on retention policies.
@@ -405,51 +365,12 @@ class DiskCache:
try: try:
with self._lock: with self._lock:
# Get snapshot of files while holding lock briefly # Get snapshot of files while holding lock briefly
entries = os.listdir(self.cache_dir) filenames = [f for f in os.listdir(self.cache_dir) if f.endswith('.json')]
except OSError as list_error: except OSError as list_error:
self.logger.error("Error listing cache directory %s: %s", self.cache_dir, list_error, exc_info=True) self.logger.error("Error listing cache directory %s: %s", self.cache_dir, list_error, exc_info=True)
stats['errors'] += 1 stats['errors'] += 1
return stats return stats
filenames = [f for f in entries if f.endswith('.json')]
# Sweep temp files abandoned by a write that never finished. set()
# removes its own in a finally, so these are the ones where the
# process died between mkstemp and os.replace -- a SIGKILL, a lost
# restart race, a power cut. Nothing ever collected them: they are
# named ".<key>.json.<random>", and the scan above only matches
# names ending in .json, so they accumulated indefinitely. Measured
# on a live rig: 76 files, 1,050 MB, 81% of the whole cache
# directory, the oldest six months old.
stats['orphan_temp_files_deleted'] = 0
for filename in (f for f in entries if self._is_orphaned_temp(f)):
# Counted as scanned like any other candidate, so files_deleted
# can never exceed files_scanned and the summary line reads
# honestly ("77/8864", not "77/0").
stats['files_scanned'] += 1
path = os.path.join(self.cache_dir, filename)
try:
# An in-flight write lives for milliseconds, so anything
# this old is certainly abandoned rather than in progress.
if (current_time - os.path.getmtime(path)) <= _ORPHAN_TEMP_MAX_AGE_SECONDS:
continue
with self._lock:
size = os.path.getsize(path)
os.remove(path)
stats['files_deleted'] += 1
stats['orphan_temp_files_deleted'] += 1
stats['space_freed_bytes'] += size
except FileNotFoundError:
continue # another sweep got there first
except OSError as e:
stats['errors'] += 1
self.logger.warning("Error deleting orphaned temp file %s: %s", filename, e)
if stats['orphan_temp_files_deleted']:
self.logger.info(
"Removed %d abandoned cache temp file(s)",
stats['orphan_temp_files_deleted'])
# Process files outside the lock to avoid blocking get/set operations # Process files outside the lock to avoid blocking get/set operations
for filename in filenames: for filename in filenames:
stats['files_scanned'] += 1 stats['files_scanned'] += 1
-10
View File
@@ -57,16 +57,6 @@ class MemoryCache:
if timestamp is None: if timestamp is None:
return None return None
# An explicit per-entry ttl wins over the caller's max_age, matching
# DiskCache. max_age is inferred from substrings in the key and is
# only a fallback for records that did not say what they wanted.
record = self._cache[key]
if isinstance(record, dict):
stored_ttl = record.get('ttl')
if isinstance(stored_ttl, (int, float)) and not isinstance(stored_ttl, bool) \
and stored_ttl >= 0:
max_age = stored_ttl
# Check expiration # Check expiration
if max_age is not None and (now - timestamp) > max_age: if max_age is not None and (now - timestamp) > max_age:
# Expired - remove it # Expired - remove it
+3 -44
View File
@@ -47,20 +47,6 @@ from src.cache.disk_cache import DateTimeEncoder # noqa: F401 - deliberate re-e
class CacheManager: class CacheManager:
"""Manages caching of API responses to reduce API calls.""" """Manages caching of API responses to reduce API calls."""
# Which cache directories already have a cleanup thread in this process.
#
# The sweep is directory-scoped work -- it lists a directory and deletes
# from it -- so one per directory is the right number no matter how many
# managers exist. Nothing enforced that before: every instance started its
# own, and because the loop closes over `self`, a discarded manager could
# never be collected and its thread woke to re-scan the same directory
# every 24 hours for the life of the process. Startup validation runs
# twice and built a throwaway manager each time, so a display process
# carried three threads for one cache.
_cleanup_owners: Dict[str, 'CacheManager'] = {}
_cleanup_owners_lock = threading.Lock()
def __init__(self) -> None: def __init__(self) -> None:
# Initialize logger first # Initialize logger first
self.logger: logging.Logger = get_logger(__name__) self.logger: logging.Logger = get_logger(__name__)
@@ -608,10 +594,8 @@ class CacheManager:
Args: Args:
key: Cache key key: Cache key
data: Data to cache data: Data to cache
ttl: Time-to-live in seconds for this entry. Takes precedence over ttl: Optional time-to-live in seconds (stored for compatibility but
the max_age a reader would otherwise apply, which is inferred expiration is still controlled via max_age when reading)
from the key and is only a fallback for entries that did not
say. Omit it to keep that inferred behaviour.
""" """
cache_data = { cache_data = {
'data': data, 'data': data,
@@ -732,29 +716,11 @@ class CacheManager:
} }
def start_cleanup_thread(self) -> None: def start_cleanup_thread(self) -> None:
"""Start background thread for periodic disk cache cleanup. """Start background thread for periodic disk cache cleanup."""
At most one thread per cache directory per process: the sweep is
directory-scoped, so a second one only duplicates the scan.
"""
if self._cleanup_thread and self._cleanup_thread.is_alive(): if self._cleanup_thread and self._cleanup_thread.is_alive():
self.logger.debug("Cleanup thread already running") self.logger.debug("Cleanup thread already running")
return return
with CacheManager._cleanup_owners_lock:
owner = CacheManager._cleanup_owners.get(self.cache_dir)
if owner is not None and owner is not self:
thread = owner._cleanup_thread
if thread is not None and thread.is_alive():
self.logger.debug(
"Cleanup thread for %s already owned by another cache "
"manager in this process; not starting a second",
self.cache_dir)
return
# The owner's thread died or was stopped -- take over.
CacheManager._cleanup_owners[self.cache_dir] = self
def cleanup_loop(): def cleanup_loop():
"""Background loop that runs cleanup periodically.""" """Background loop that runs cleanup periodically."""
self.logger.info("Disk cache cleanup thread started (interval: %d hours)", self.logger.info("Disk cache cleanup thread started (interval: %d hours)",
@@ -802,13 +768,6 @@ class CacheManager:
Signals the thread to stop and waits for it to finish (with timeout). Signals the thread to stop and waits for it to finish (with timeout).
This allows for clean shutdown during testing or application termination. This allows for clean shutdown during testing or application termination.
""" """
# Release ownership first and unconditionally, so a manager that never
# started a thread (or whose thread already exited) cannot keep the
# directory claimed and block a live manager from sweeping it.
with CacheManager._cleanup_owners_lock:
if CacheManager._cleanup_owners.get(self.cache_dir) is self:
del CacheManager._cleanup_owners[self.cache_dir]
if not self._cleanup_thread or not self._cleanup_thread.is_alive(): if not self._cleanup_thread or not self._cleanup_thread.is_alive():
self.logger.debug("Cleanup thread not running") self.logger.debug("Cleanup thread not running")
return return
+8 -80
View File
@@ -44,20 +44,6 @@ from src.common.sync_manager import DisplaySyncManager, SyncRole
# Get logger with consistent configuration # Get logger with consistent configuration
logger = get_logger(__name__) logger = get_logger(__name__)
# How long startup will wait for plugins to fetch their first data before
# showing anything. Each plugin's update blocks for up to the executor's 30s
# timeout and they run one after another, so the uncapped total is the sum of
# every slow plugin: 82 seconds on the worst boot measured, with a blank panel
# throughout. Whatever does not finish in time is picked up by the scheduled
# update tick moments later, with the display already running.
_INITIAL_UPDATE_BUDGET_SECONDS = 20.0
# The least budget worth starting a plugin with. Below this the plugin is
# deferred instead: granting it a floor would let the pass run past its
# deadline, and granting it the true remainder would record a timeout for a
# slot it never had a chance to use.
_MIN_INITIAL_UPDATE_TIMEOUT_SECONDS = 2.0
# Vegas mode import (lazy loaded to avoid circular imports) # Vegas mode import (lazy loaded to avoid circular imports)
_vegas_mode_imported = False _vegas_mode_imported = False
VegasModeCoordinator = None VegasModeCoordinator = None
@@ -104,8 +90,7 @@ class DisplayController:
# Validate startup configuration # Validate startup configuration
try: try:
from src.startup_validator import StartupValidator from src.startup_validator import StartupValidator
validator = StartupValidator(self.config_manager, validator = StartupValidator(self.config_manager)
cache_manager=self.cache_manager)
is_valid, errors, warnings = validator.validate_all() is_valid, errors, warnings = validator.validate_all()
if warnings: if warnings:
@@ -273,8 +258,7 @@ class DisplayController:
# Validate plugins after plugin manager is created # Validate plugins after plugin manager is created
try: try:
from src.startup_validator import StartupValidator from src.startup_validator import StartupValidator
validator = StartupValidator(self.config_manager, self.plugin_manager, validator = StartupValidator(self.config_manager, self.plugin_manager)
cache_manager=self.cache_manager)
is_valid, errors, warnings = validator.validate_all() is_valid, errors, warnings = validator.validate_all()
if warnings: if warnings:
@@ -477,7 +461,7 @@ class DisplayController:
# Initial data update for plugins (ensures data available on first display) # Initial data update for plugins (ensures data available on first display)
logger.info("Performing initial plugin data update...") logger.info("Performing initial plugin data update...")
update_start = time.time() update_start = time.time()
self._update_modules(deadline=update_start + _INITIAL_UPDATE_BUDGET_SECONDS) self._update_modules()
logger.info("Initial plugin update completed in %.3f seconds", time.time() - update_start) logger.info("Initial plugin update completed in %.3f seconds", time.time() - update_start)
# Initialize Vegas mode coordinator # Initialize Vegas mode coordinator
@@ -833,42 +817,14 @@ class DisplayController:
self._cached_target_brightness = normal_brightness # persist for minute-gate self._cached_target_brightness = normal_brightness # persist for minute-gate
return normal_brightness return normal_brightness
def _update_modules(self, deadline: Optional[float] = None): def _update_modules(self):
"""Update all plugin modules. """Update all plugin modules."""
Args:
deadline: Wall-clock time after which remaining plugins are left
for the scheduled update tick instead of being waited on. Each
update blocks this thread for up to the executor's timeout, and
they run one after another, so without a bound the total is the
sum of every slow plugin on the system. Measured at startup on
a live rig: 82 seconds, 55 and 26 on the two boots before -- all
of it with nothing on the panel.
"""
if not self.plugin_manager: if not self.plugin_manager:
return return
# Update all loaded plugins # Update all loaded plugins
plugins_dict = getattr(self.plugin_manager, 'loaded_plugins', None) or getattr(self.plugin_manager, 'plugins', {}) plugins_dict = getattr(self.plugin_manager, 'loaded_plugins', None) or getattr(self.plugin_manager, 'plugins', {})
deferred = []
for plugin_id, plugin_instance in plugins_dict.items(): for plugin_id, plugin_instance in plugins_dict.items():
update_timeout = None
if deadline is not None:
update_timeout = deadline - time.time()
if update_timeout < _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS:
# Too little left to be worth starting. Deferring rather
# than granting a floor keeps the budget a real ceiling --
# clamping up to a minimum let a plugin that began with a
# sliver left run on past the deadline -- and a plugin
# handed a slot it cannot use would just be recorded as
# having timed out.
#
# Nothing is lost either way: a plugin that has never
# updated is immediately due, so run_scheduled_updates()
# picks it up within seconds, with the display already
# running.
deferred.append(plugin_id)
continue
# Check circuit breaker before attempting update # Check circuit breaker before attempting update
if hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker: if hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
if self.plugin_manager.health_tracker.should_skip_plugin(plugin_id): if self.plugin_manager.health_tracker.should_skip_plugin(plugin_id):
@@ -877,13 +833,7 @@ class DisplayController:
# Use PluginExecutor if available for safe execution # Use PluginExecutor if available for safe execution
if hasattr(self.plugin_manager, 'plugin_executor'): if hasattr(self.plugin_manager, 'plugin_executor'):
# The remaining budget is the timeout, so the pass cannot success = self.plugin_manager.plugin_executor.execute_update(plugin_instance, plugin_id)
# run past its deadline. Bounding the loop alone did not do
# it: the last plugin to start could still block for the
# executor's full 30s, which turned a 20s budget into a 31.8s
# pass on the rig.
success = self.plugin_manager.plugin_executor.execute_update(
plugin_instance, plugin_id, timeout=update_timeout)
if success and hasattr(self.plugin_manager, 'plugin_last_update'): if success and hasattr(self.plugin_manager, 'plugin_last_update'):
self.plugin_manager.plugin_last_update[plugin_id] = time.time() self.plugin_manager.plugin_last_update[plugin_id] = time.time()
else: else:
@@ -902,12 +852,6 @@ class DisplayController:
if hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker: if hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
self.plugin_manager.health_tracker.record_failure(plugin_id, exc) self.plugin_manager.health_tracker.record_failure(plugin_id, exc)
if deferred:
logger.info(
"Initial update budget spent; %d plugin(s) left to the update "
"tick so the display can start: %s",
len(deferred), ", ".join(deferred))
def _tick_plugin_updates_for_vegas(self) -> None: def _tick_plugin_updates_for_vegas(self) -> None:
"""Run scheduled plugin updates and tell Vegas mode which plugins """Run scheduled plugin updates and tell Vegas mode which plugins
actually got fresh data, so it can hot-swap them into the scroll actually got fresh data, so it can hot-swap them into the scroll
@@ -1694,12 +1638,6 @@ class DisplayController:
logger.warning("Error checking live priority for %s: %s", mode_name, e) logger.warning("Error checking live priority for %s: %s", mode_name, e)
return live return live
def _vegas_keeps_live_in_ticker(self) -> bool:
"""Whether live content should stay in the ticker instead of preempting it."""
coordinator = getattr(self, 'vegas_coordinator', None)
config = getattr(coordinator, 'vegas_config', None)
return bool(getattr(config, 'live_in_ticker', False))
def _check_live_priority(self, advance=False): def _check_live_priority(self, advance=False):
"""Return the live-priority mode to display, or None if nothing is live. """Return the live-priority mode to display, or None if nothing is live.
@@ -1913,24 +1851,14 @@ class DisplayController:
# Check for live priority content and switch to it immediately. # Check for live priority content and switch to it immediately.
# advance=True so multiple simultaneously-live games take turns # advance=True so multiple simultaneously-live games take turns
# (round-robin) instead of pinning to the first plugin. # (round-robin) instead of pinning to the first plugin.
# Skipped when the ticker is keeping live content: switching if not self.on_demand_active and not wifi_status_data:
# the rotation underneath Vegas would move current_mode_index
# and stash a resume point for a takeover that never happens.
if (not self.on_demand_active and not wifi_status_data
and not (self._is_vegas_mode_active()
and self._vegas_keeps_live_in_ticker())):
live_priority_mode = self._check_live_priority(advance=True) live_priority_mode = self._check_live_priority(advance=True)
self._apply_live_priority(live_priority_mode) self._apply_live_priority(live_priority_mode)
# Vegas scroll mode - continuous ticker across all plugins # Vegas scroll mode - continuous ticker across all plugins
# Priority: on-demand > wifi-status > live-priority > vegas > normal rotation # Priority: on-demand > wifi-status > live-priority > vegas > normal rotation
if self._is_vegas_mode_active() and not wifi_status_data: if self._is_vegas_mode_active() and not wifi_status_data:
# Live content normally preempts the ticker entirely. With live_mode = self._check_live_priority()
# vegas_scroll.live_in_ticker the marquee keeps running and
# the live plugin takes extra turns inside it instead --
# see StreamManager._apply_priority_weights.
live_mode = (None if self._vegas_keeps_live_in_ticker()
else self._check_live_priority())
if not live_mode: if not live_mode:
try: try:
# Run Vegas mode iteration # Run Vegas mode iteration
+3 -112
View File
@@ -25,7 +25,6 @@ the same object.
import json import json
import os import os
import socket
import tempfile import tempfile
if os.getenv("EMULATOR", "false") == "true": if os.getenv("EMULATOR", "false") == "true":
from RGBMatrixEmulator import RGBMatrix, RGBMatrixOptions from RGBMatrixEmulator import RGBMatrix, RGBMatrixOptions
@@ -259,26 +258,6 @@ class DisplayManager:
# Initialize managers # Initialize managers
# Calendar manager is now initialized by DisplayController # Calendar manager is now initialized by DisplayController
# Orientation setting -> rpi-rgb-led-matrix "Rotate:<deg>" pixel-mapper suffix.
# "normal" needs no suffix since 0 degrees is the identity transform.
_ORIENTATION_ROTATE_DEGREES = {'normal': None, '90': 90, '180': 180, '270': 270}
def _build_pixel_mapper_config(self, hardware_config: dict) -> str:
"""Compose the raw pixel_mapper_config string with the orientation setting.
`pixel_mapper_config` stays available as a free-form advanced field (e.g.
for "U-mapper" chain layouts); `orientation` is the user-facing dropdown
for physical mounting (e.g. panels mounted upside down) and is appended as
a "Rotate:<deg>" mapper rather than overwriting any existing config.
"""
base_mapper = (hardware_config.get('pixel_mapper_config') or '').strip()
orientation = hardware_config.get('orientation', 'normal')
degrees = self._ORIENTATION_ROTATE_DEGREES.get(orientation)
if degrees is None:
return base_mapper
rotate_mapper = f'Rotate:{degrees}'
return f'{base_mapper};{rotate_mapper}' if base_mapper else rotate_mapper
def _setup_matrix(self): def _setup_matrix(self):
"""Initialize the RGB matrix with configuration settings.""" """Initialize the RGB matrix with configuration settings."""
_init_error_str = None _init_error_str = None
@@ -304,7 +283,7 @@ class DisplayManager:
options.pwm_bits = hardware_config.get('pwm_bits', 10) options.pwm_bits = hardware_config.get('pwm_bits', 10)
options.pwm_lsb_nanoseconds = hardware_config.get('pwm_lsb_nanoseconds', 150) options.pwm_lsb_nanoseconds = hardware_config.get('pwm_lsb_nanoseconds', 150)
options.led_rgb_sequence = hardware_config.get('led_rgb_sequence', 'RGB') options.led_rgb_sequence = hardware_config.get('led_rgb_sequence', 'RGB')
options.pixel_mapper_config = self._build_pixel_mapper_config(hardware_config) options.pixel_mapper_config = hardware_config.get('pixel_mapper_config', '')
options.row_address_type = hardware_config.get('row_address_type', 0) options.row_address_type = hardware_config.get('row_address_type', 0)
options.multiplexing = hardware_config.get('multiplexing', 0) options.multiplexing = hardware_config.get('multiplexing', 0)
options.panel_type = hardware_config.get('panel_type', '') options.panel_type = hardware_config.get('panel_type', '')
@@ -518,91 +497,6 @@ class DisplayManager:
logger.warning(f"[BRIGHTNESS] Matrix does not support brightness property: {e}", exc_info=True) logger.warning(f"[BRIGHTNESS] Matrix does not support brightness property: {e}", exc_info=True)
return -1 return -1
@staticmethod
def _local_ip() -> Optional[str]:
"""This device's address on the network it routes through, or None.
Deliberately not `hostname -I` or a systemctl probe for AP mode, which
is how the web launcher does it: both spawn processes with multi-second
timeouts, and this runs on the startup path the rest of this change
exists to shorten. Connecting a UDP socket sends no packets -- it only
asks the kernel which source address it would use -- so it costs
microseconds and works with the network down, as long as a route
exists.
"""
sock = None
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.settimeout(0.2)
sock.connect(("8.8.8.8", 80)) # nosec B104 - no traffic; selects a route
ip = sock.getsockname()[0]
return ip if ip and not ip.startswith("127.") else None
except OSError:
return None
finally:
if sock is not None:
try:
sock.close()
except OSError:
pass
def _fitting_font(self, lines, width):
"""The largest font from the usual ladder that fits every line."""
candidates = [self.font,
("assets/fonts/4x6-font.ttf", 6)]
for candidate in candidates:
try:
font = candidate
if isinstance(candidate, tuple):
font = ImageFont.truetype(candidate[0], candidate[1])
if all(self.draw.textlength(t, font=font) <= width for t in lines):
return font
except (OSError, ValueError, AttributeError):
continue
return self.font
def _draw_startup_banner(self, lines, width: int, height: int) -> None:
"""Centre `lines` over whatever the test pattern already drew.
This screen stays on the panel for the whole initial plugin update, and
on a headless Pi it is the only place the device's address appears
without going looking for it -- so it has to be readable off a wall,
not merely present.
The font is chosen to fit rather than fixed at 8px: "Initializing" is
96px in PressStart2P, which ran off the side of a 64px panel even
before an address was added. And the pattern is punched out behind the
text, because the diagonal runs through the middle of the panel, which
is exactly where this sits.
The text stays blue. It is not decoration: the pattern draws one pure
channel per element -- red border, green diagonal, blue text -- so that
a glance at the panel says whether led_rgb_sequence is right. Swap the
wiring to BGR and the border comes up blue and this text red. Drawing
it white would light all three channels and destroy the only blue
reference on the screen, which is why it is worth a comment rather
than a quiet preference.
"""
if not lines:
return
font = self._fitting_font(lines, width - 2)
line_height = self.draw.textbbox((0, 0), "Ag", font=font)[3] + 1
block_height = line_height * len(lines)
block_top = max(1, (height - block_height) // 2)
block_width = max(self.draw.textlength(t, font=font) for t in lines)
block_left = max(0, (width - block_width) // 2)
self.draw.rectangle(
[block_left - 2, block_top - 1,
block_left + block_width + 1, block_top + block_height],
fill=(0, 0, 0))
for row, line in enumerate(lines):
line_width = self.draw.textlength(line, font=font)
self.draw.text(
(max(0, (width - line_width) // 2), block_top + row * line_height),
line, font=font, fill=(0, 0, 255))
def _draw_test_pattern(self): def _draw_test_pattern(self):
"""Draw a test pattern to verify the display is working.""" """Draw a test pattern to verify the display is working."""
try: try:
@@ -622,11 +516,8 @@ class DisplayManager:
# Draw a diagonal line # Draw a diagonal line
self.draw.line([0, 0, self.matrix.width-1, self.matrix.height-1], fill=(0, 255, 0)) self.draw.line([0, 0, self.matrix.width-1, self.matrix.height-1], fill=(0, 255, 0))
lines = ["Initializing"] # Draw some text - changed from "TEST" to "Initializing" with smaller font
ip = self._local_ip() self.draw.text((10, 10), "Initializing", font=self.font, fill=(0, 0, 255))
if ip:
lines.append(ip)
self._draw_startup_banner(lines, self.matrix.width, self.matrix.height)
# Update the display once after everything is drawn # Update the display once after everything is drawn
self.update_display() self.update_display()
-42
View File
@@ -555,48 +555,6 @@ class BasePlugin(ABC):
""" """
return False return False
def get_vegas_priority_weight(self) -> Optional[int]:
"""How many slots per Vegas cycle this plugin should get, or None.
The Vegas ticker is otherwise a strict round robin: every plugin
appears exactly once per cycle. With a dozen plugins enabled that puts
minutes between a live score and its next appearance. A weight of N
gives the plugin N slots per cycle, spread evenly through it rather
than clumped together.
Return ``None`` (the default) to let the core decide. It gives a
plugin ``vegas_scroll.live_weight`` when ``has_live_priority()`` and
``has_live_content()`` are both true, and 1 otherwise -- so live sports
already get extra turns without implementing this at all.
Implement it only when the plugin knows something the core cannot. The
motivating case is favorite teams: the core can see *that* a game is
live but not *whose*, so a scoreboard that wants its favorite's game
shown more often than other live games has to say so::
def get_vegas_priority_weight(self):
if not (self.has_live_priority() and self.has_live_content()):
return None # let the core decide
cfg = self.global_config.get('display', {}).get('vegas_scroll', {})
if self._favorite_is_live():
return cfg.get('favorite_live_weight', 5)
return cfg.get('live_weight', 3)
The weight is per *plugin*, not per game. A scoreboard showing four
live games still occupies one slot at a time and rotates its own games
within that slot; this controls how often the plugin itself comes
round.
Raising is safe: the core logs it and falls back to its own
live-content check, so a broken weight calculation costs the plugin
the favorite distinction but not the live boost.
Returns:
Slots per cycle (clamped to 1..10 by the caller), or None to
defer to the core's own live-content weighting.
"""
return None
def get_live_modes(self) -> List[str]: def get_live_modes(self) -> List[str]:
""" """
Get list of display modes that should be used during live priority takeover. Get list of display modes that should be used during live priority takeover.
+1 -20
View File
@@ -15,23 +15,16 @@ from src.logging_config import get_logger
class StartupValidator: class StartupValidator:
"""Validates system state on startup.""" """Validates system state on startup."""
def __init__(self, config_manager: Any, plugin_manager: Optional[Any] = None, def __init__(self, config_manager: Any, plugin_manager: Optional[Any] = None) -> None:
cache_manager: Optional[Any] = None) -> None:
""" """
Initialize the startup validator. Initialize the startup validator.
Args: Args:
config_manager: ConfigManager instance config_manager: ConfigManager instance
plugin_manager: Optional PluginManager instance plugin_manager: Optional PluginManager instance
cache_manager: The CacheManager the application will actually use.
Pass it. Without one this validator builds its own just to read
a directory path, which reports on a cache the app does not
use and leaves behind a cleanup thread that nothing stops --
validation runs twice per startup, so that was two of them.
""" """
self.config_manager = config_manager self.config_manager = config_manager
self.plugin_manager = plugin_manager self.plugin_manager = plugin_manager
self.cache_manager = cache_manager
self.logger = get_logger(__name__) self.logger = get_logger(__name__)
self.errors: List[str] = [] self.errors: List[str] = []
self.warnings: List[str] = [] self.warnings: List[str] = []
@@ -98,20 +91,8 @@ class StartupValidator:
def _validate_cache_directory(self) -> None: def _validate_cache_directory(self) -> None:
"""Validate cache directory permissions.""" """Validate cache directory permissions."""
try: try:
cache_manager = self.cache_manager
if cache_manager is None:
# No caller supplied one (older embedders, direct use in a
# script). Build one, but do not leave its cleanup thread
# running behind us -- this instance is discarded on the next
# line but the thread is a closure over it, so it would never
# be collected.
from src.cache_manager import CacheManager from src.cache_manager import CacheManager
cache_manager = CacheManager() cache_manager = CacheManager()
try:
cache_dir = cache_manager.get_cache_dir()
finally:
cache_manager.stop_cleanup_thread()
else:
cache_dir = cache_manager.get_cache_dir() cache_dir = cache_manager.get_cache_dir()
if not cache_dir: if not cache_dir:
-44
View File
@@ -125,32 +125,6 @@ class VegasModeConfig:
plugin_order: List[str] = field(default_factory=list) plugin_order: List[str] = field(default_factory=list)
excluded_plugins: Set[str] = field(default_factory=set) excluded_plugins: Set[str] = field(default_factory=set)
# --- Live content in the ticker -------------------------------------
#
# By default a live game preempts Vegas entirely: the display controller
# refuses to run the ticker while any plugin reports live priority, and you
# get the full-screen scoreboard instead. Set live_in_ticker to keep the
# marquee running and let live content take extra turns within it.
#
# The rotation is otherwise a strict round robin -- every plugin appears
# exactly once per cycle -- so with a dozen plugins enabled a live score
# comes round once a lap and can be minutes old on screen. Weighting lets a
# plugin claim several slots per cycle instead.
#
# Weights are per plugin, not per game: a scoreboard showing four live
# games still occupies one slot at a time, and rotates its own games within
# that slot using its own favorite_live_boost.
live_in_ticker: bool = False
# Slots per cycle for a plugin reporting live content. 1 disables the boost
# and restores the plain round robin.
live_weight: int = 3
# Slots per cycle for a plugin whose live content involves a favorite team.
# Only plugins implementing get_vegas_priority_weight() can claim this --
# the core cannot tell whose game is on, so the plugin reports it.
favorite_live_weight: int = 5
# Performance settings # Performance settings
target_fps: int = 125 # Target frame rate target_fps: int = 125 # Target frame rate
buffer_ahead: int = 2 # Number of plugins to buffer ahead buffer_ahead: int = 2 # Number of plugins to buffer ahead
@@ -201,12 +175,6 @@ class VegasModeConfig:
overflow_mode=str(vegas_config.get('overflow_mode', 'rotate')), overflow_mode=str(vegas_config.get('overflow_mode', 'rotate')),
plugin_order=list(vegas_config.get('plugin_order', [])), plugin_order=list(vegas_config.get('plugin_order', [])),
excluded_plugins=set(vegas_config.get('excluded_plugins', [])), excluded_plugins=set(vegas_config.get('excluded_plugins', [])),
live_in_ticker=bool(vegas_config.get('live_in_ticker', False)),
# Clamped: a weight below 1 would drop the plugin from the rotation
# entirely, and a very large one starves everything else.
live_weight=max(1, min(10, int(vegas_config.get('live_weight', 3)))),
favorite_live_weight=max(
1, min(10, int(vegas_config.get('favorite_live_weight', 5)))),
target_fps=int(vegas_config.get('target_fps', 125)), target_fps=int(vegas_config.get('target_fps', 125)),
buffer_ahead=int(vegas_config.get('buffer_ahead', 2)), buffer_ahead=int(vegas_config.get('buffer_ahead', 2)),
frame_based_scrolling=vegas_config.get('frame_based_scrolling', True), frame_based_scrolling=vegas_config.get('frame_based_scrolling', True),
@@ -236,9 +204,6 @@ class VegasModeConfig:
'lead_in_width': self.lead_in_width, 'lead_in_width': self.lead_in_width,
'plugins_per_cycle': self.plugins_per_cycle, 'plugins_per_cycle': self.plugins_per_cycle,
'max_plugin_width_ratio': self.max_plugin_width_ratio, 'max_plugin_width_ratio': self.max_plugin_width_ratio,
'live_in_ticker': self.live_in_ticker,
'live_weight': self.live_weight,
'favorite_live_weight': self.favorite_live_weight,
'overflow_mode': self.overflow_mode, 'overflow_mode': self.overflow_mode,
'plugin_order': self.plugin_order, 'plugin_order': self.plugin_order,
'excluded_plugins': list(self.excluded_plugins), 'excluded_plugins': list(self.excluded_plugins),
@@ -406,15 +371,6 @@ class VegasModeConfig:
if 'enabled' in vegas_config: if 'enabled' in vegas_config:
self.enabled = vegas_config['enabled'] self.enabled = vegas_config['enabled']
if 'live_in_ticker' in vegas_config:
self.live_in_ticker = bool(vegas_config['live_in_ticker'])
# Clamped exactly as from_config does: a weight below 1 would drop the
# plugin from the rotation, and a huge one starves everything else.
if 'live_weight' in vegas_config:
self.live_weight = max(1, min(10, int(vegas_config['live_weight'])))
if 'favorite_live_weight' in vegas_config:
self.favorite_live_weight = max(
1, min(10, int(vegas_config['favorite_live_weight'])))
if 'scroll_speed' in vegas_config: if 'scroll_speed' in vegas_config:
self.scroll_speed = float(vegas_config['scroll_speed']) self.scroll_speed = float(vegas_config['scroll_speed'])
if 'separator_width' in vegas_config: if 'separator_width' in vegas_config:
+2 -39
View File
@@ -12,7 +12,6 @@ Supports three display modes per plugin:
""" """
import logging import logging
import math
import time import time
import threading import threading
from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING
@@ -31,21 +30,6 @@ if TYPE_CHECKING:
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _percentile(ordered: List[float], fraction: float) -> float:
"""Nearest-rank percentile of an already-sorted list.
Index ceil(n * fraction) - 1, so 100 samples at 0.99 give the 99th-ranked
value. The obvious int(n * fraction) is off by one and, at exactly 100
samples, lands on the maximum -- which is the number already reported
alongside this one as the worst frame, so the two columns would agree
precisely when the sample was smallest.
"""
if not ordered:
return 0.0
index = math.ceil(len(ordered) * fraction) - 1
return ordered[min(len(ordered) - 1, max(0, index))]
class VegasModeCoordinator: class VegasModeCoordinator:
""" """
Orchestrates Vegas scroll mode operation. Orchestrates Vegas scroll mode operation.
@@ -398,12 +382,6 @@ class VegasModeCoordinator:
fps_log_interval = 5.0 # Log FPS every 5 seconds fps_log_interval = 5.0 # Log FPS every 5 seconds
last_fps_log_time = start_time last_fps_log_time = start_time
fps_frame_count = 0 fps_frame_count = 0
# A mean hides stutter completely. At 120fps a five-second window is
# ~600 frames, so a 200ms freeze -- plainly visible on a marquee --
# moves the average from 120.0 to 115.4 and reads as healthy. What a
# viewer actually notices is the worst frame, so track that too.
frame_worst = 0.0
frame_times: List[float] = []
logger.info("Starting Vegas iteration for %.1fs", duration) logger.info("Starting Vegas iteration for %.1fs", duration)
@@ -439,11 +417,6 @@ class VegasModeCoordinator:
frame_elapsed = time.time() - frame_started frame_elapsed = time.time() - frame_started
time.sleep(max(0.0, frame_interval - frame_elapsed)) time.sleep(max(0.0, frame_interval - frame_elapsed))
# Measured before the sleep: time spent working, not pacing.
if frame_elapsed > frame_worst:
frame_worst = frame_elapsed
frame_times.append(frame_elapsed)
# Increment frame count and check for interrupt periodically # Increment frame count and check for interrupt periodically
frame_count += 1 frame_count += 1
fps_frame_count += 1 fps_frame_count += 1
@@ -452,16 +425,12 @@ class VegasModeCoordinator:
current_time = time.time() current_time = time.time()
if current_time - last_fps_log_time >= fps_log_interval: if current_time - last_fps_log_time >= fps_log_interval:
fps = fps_frame_count / (current_time - last_fps_log_time) fps = fps_frame_count / (current_time - last_fps_log_time)
p99 = _percentile(sorted(frame_times), 0.99)
logger.info( logger.info(
"Vegas FPS: %.1f (target: %d, frames: %d) p99 %.1fms worst %.1fms", "Vegas FPS: %.1f (target: %d, frames: %d)",
fps, self.vegas_config.target_fps, fps_frame_count, fps, self.vegas_config.target_fps, fps_frame_count
p99 * 1000.0, frame_worst * 1000.0
) )
last_fps_log_time = current_time last_fps_log_time = current_time
fps_frame_count = 0 fps_frame_count = 0
frame_worst = 0.0
frame_times.clear()
if (self._interrupt_check and if (self._interrupt_check and
frame_count % self._interrupt_check_interval == 0): frame_count % self._interrupt_check_interval == 0):
@@ -528,12 +497,6 @@ class VegasModeCoordinator:
if not self._live_priority_check: if not self._live_priority_check:
return False return False
if self.vegas_config.live_in_ticker:
# The ticker keeps live content rather than yielding to it; the
# extra turns are arranged in the rotation itself, so there is
# nothing to pause for.
return False
try: try:
live_mode = self._live_priority_check() live_mode = self._live_priority_check()
if live_mode: if live_mode:
-139
View File
@@ -406,8 +406,6 @@ class StreamManager:
) )
logger.info("Ordered plugins: %s", ordered_plugins) logger.info("Ordered plugins: %s", ordered_plugins)
ordered_plugins = self._apply_priority_weights(ordered_plugins)
# Atomically update shared state under lock to avoid races with prefetchers # Atomically update shared state under lock to avoid races with prefetchers
with self._buffer_lock: with self._buffer_lock:
self._ordered_plugins = ordered_plugins self._ordered_plugins = ordered_plugins
@@ -419,143 +417,6 @@ class StreamManager:
logger.info("=" * 60) logger.info("=" * 60)
def _plugin_weight(self, plugin_id: str) -> int:
"""Slots per cycle for one plugin.
A plugin may answer for itself via get_vegas_priority_weight() -- the
only way favorite-team awareness can reach here, since the core can see
that a game is live but not whose. When it declines (returns None, the
default), live content earns ``live_weight`` and everything else 1.
"""
plugin = None
try:
plugin = self.plugin_manager.plugins.get(plugin_id)
except (AttributeError, TypeError):
return 1
if plugin is None:
return 1
try:
if hasattr(plugin, 'get_vegas_priority_weight'):
declared = plugin.get_vegas_priority_weight()
if declared is not None:
return max(1, min(10, int(declared)))
except Exception:
# Deliberately falls through to the core's own live check rather
# than demoting to 1. The plugin's weight calculation is broken,
# but has_live_priority() and has_live_content() are separate
# methods guarded separately below -- a plugin that genuinely has
# a live game should still get live_weight for it.
logger.exception("[%s] get_vegas_priority_weight() failed", plugin_id)
try:
if (hasattr(plugin, 'has_live_priority')
and hasattr(plugin, 'has_live_content')
and plugin.has_live_priority()
and plugin.has_live_content()):
return self.config.live_weight
except Exception:
logger.exception("[%s] live-content check failed", plugin_id)
return 1
def _apply_priority_weights(self, ordered: List[str]) -> List[str]:
"""Expand the rotation so weighted plugins take several turns per cycle.
Smooth Weighted Round-Robin, the same scheduler the sports plugins use
to rotate their own games: a plugin of weight N appears N times per
cycle, and the repeats are spaced through the cycle rather than
clumped, so a live score is never three-in-a-row followed by a long
silence.
Returns the input unchanged when nothing is weighted, which is both the
common case and the pre-existing behaviour.
"""
if not ordered or not self.config.live_in_ticker:
return ordered
weights = {pid: self._plugin_weight(pid) for pid in ordered}
total = sum(weights.values())
if total <= len(ordered):
return ordered # nothing boosted; plain round robin
current = {pid: 0 for pid in ordered}
schedule: List[str] = []
for _ in range(total):
for pid in ordered:
current[pid] += weights[pid]
picked = max(current, key=lambda p: current[p])
current[picked] -= total
schedule.append(picked)
schedule = self._unclump_seam(schedule)
boosted = {p: w for p, w in weights.items() if w > 1}
logger.info(
"Vegas rotation weighted: %d slots for %d plugins (boosted: %s)",
len(schedule), len(ordered), boosted)
return schedule
@staticmethod
def _unclump_seam(schedule: List[str]) -> List[str]:
"""Stop the heaviest plugin sitting on both ends of the cycle.
Smooth Weighted Round-Robin spaces repeats well *within* a pass, but
it schedules the heaviest item first and often last too. The strip
loops, so those two are neighbours: the one place the marquee shows
the same plugin twice running is the seam between cycles.
Rotating the list cannot fix this. Rotation preserves the cyclic order
exactly, so it only moves where the seam is drawn, not the adjacency
itself. The trailing entry has to be swapped with one from the middle
whose neighbours differ from it, which breaks the pair without
creating another.
Left alone when no such position exists -- a rotation short enough or
lopsided enough to have none is one where the plugin is unavoidably
adjacent to itself anyway.
"""
if len(schedule) < 3 or schedule[0] != schedule[-1]:
return schedule
repeated = schedule[-1]
size = len(schedule)
def cyclic_doubles(seq) -> int:
return sum(1 for i in range(size) if seq[i] == seq[(i + 1) % size])
def clearance(seq, value) -> int:
"""Smallest cyclic gap between appearances of `value`."""
at = [i for i, v in enumerate(seq) if v == value]
if len(at) < 2:
return size
return min(min((b - a) % size, (a - b) % size)
for i, a in enumerate(at) for b in at[i + 1:])
# Try each swap and judge the result, rather than reasoning about which
# neighbours the two moved elements will end up with. That reasoning is
# where the first version went wrong: it guarded the slot `repeated`
# moves into but not the one the displaced element lands in, so
# ['a','b','c','d','x','y','x','a'] came back ending ['x','x'] -- the
# seam duplicate traded for a fresh one.
best = None
best_clearance = -1
for j in range(1, size - 1):
candidate = list(schedule)
candidate[j], candidate[-1] = candidate[-1], candidate[j]
if cyclic_doubles(candidate):
continue
# Among the repairs that work, prefer the one that leaves the
# boosted plugin most evenly spread; taking the first that merely
# fits moved a repeat from a gap of 7 into a gap of 2.
spread = clearance(candidate, repeated)
if spread > best_clearance:
best, best_clearance = candidate, spread
# None exists when the value is unavoidably adjacent to itself -- a
# plugin holding most of the slots has to be. Schedule it as it is
# rather than refuse.
return best if best is not None else schedule
def _prefetch_content(self, count: int = 1) -> None: def _prefetch_content(self, count: int = 1) -> None:
""" """
Prefetch content for upcoming plugins. Prefetch content for upcoming plugins.
-92
View File
@@ -4,7 +4,6 @@ Centralized error handling for web interface.
Provides helpers for consistent error responses across API endpoints. Provides helpers for consistent error responses across API endpoints.
""" """
import re
from typing import Any, Optional from typing import Any, Optional
from flask import jsonify from flask import jsonify
@@ -17,97 +16,6 @@ from src.logging_config import get_logger
logger = get_logger(__name__) logger = get_logger(__name__)
# Credentials that turn up inside exception text. A requests error quotes the
# URL it failed on, and plugins that authenticate by query string put their key
# there, so echoing an exception verbatim can hand out an API key. Redact the
# value, keep the parameter name -- knowing *which* credential was involved is
# part of the diagnosis.
_REDACT_CREDENTIAL = re.compile(
r'((?:api[_-]?key|access[_-]?token|auth|apikey|key|passwd|password|pwd|'
r'secret|sig|signature|token)["\']?\s*[=:]\s*["\']?)([^\s&"\'<>,}]+)',
re.IGNORECASE,
)
# `Authorization: <scheme> <credential>`. The scheme name is kept because it
# says which kind of credential failed; the credential goes. Any scheme
# matches, not a fixed list: ApiKey, Negotiate, NTLM, AWS4-HMAC-SHA256 and
# whatever a plugin's API invents next are all credentials, and a list would
# silently leak the ones nobody thought of. Not covered by the generic pattern
# above, whose value part stops at whitespace and so would keep the credential
# once a space follows the scheme.
_REDACT_AUTH_HEADER = re.compile(
r'((?:proxy-)?authorization["\']?\s*[=:]\s*["\']?\s*'
r'(?:[A-Za-z][\w.+-]*[ \t]+)?)' # optional scheme name, kept
r'([^\s,"\'<>}]+)', # the credential, redacted
re.IGNORECASE,
)
# Credentials embedded in a URL: https://user:password@host. requests quotes
# the full URL in its exceptions, so this is a realistic leak. The username is
# kept -- it identifies which account failed without being the secret.
_REDACT_URL_USERINFO = re.compile(r'([a-z][a-z0-9+.-]*://[^/\s:@]+:)([^/\s@]+)(@)',
re.IGNORECASE)
# Long enough for an errno string with a path, short enough not to dump a
# parser's worth of context into a JSON field.
_MAX_DETAIL_LENGTH = 400
def describe_exception(exc: BaseException,
max_length: int = _MAX_DETAIL_LENGTH) -> str:
"""
One-line, safe-to-return description of an exception.
The generic "an error occurred; see logs for details" tells a user nothing
and, when the failure is bad enough, the logs are unreachable too: a device
whose storage was failing returned that message from every endpoint
*including* the log viewer, because journalctl could not be executed. The
underlying `[Errno 5] Input/output error` named the fault immediately.
Returns "TypeName: message", credentials redacted and length capped. The
type alone is worth carrying -- a bare PermissionError says more than any
generic sentence.
Args:
exc: The exception to describe
max_length: Truncate beyond this many characters
Returns:
A single-line description, never empty
"""
message = str(exc).strip()
text = f"{type(exc).__name__}: {message}" if message else type(exc).__name__
return redact_text(text, max_length)
def redact_text(text: str, max_length: int = _MAX_DETAIL_LENGTH) -> str:
"""Make arbitrary text safe to hand back over HTTP.
Split out of describe_exception because exceptions are not the only thing
worth returning: a subprocess's stderr, or a message a helper script
printed, is just as useful to a user and just as capable of carrying a
token or a password in it.
Args:
text: The text to redact
max_length: Truncate beyond this many characters
Returns:
A single line, credentials replaced, length capped.
"""
text = text or ''
# Order matters: the URL and header forms are more specific than the
# generic key=value pattern, which would otherwise chew the scheme.
text = _REDACT_URL_USERINFO.sub(r'\1<redacted>\3', text)
text = _REDACT_AUTH_HEADER.sub(r'\1<redacted>', text)
text = _REDACT_CREDENTIAL.sub(r'\1<redacted>', text)
# Collapse newlines/tabs so the detail stays one line in a JSON field.
text = ' '.join(text.split())
if len(text) > max_length:
text = text[:max_length - 1].rstrip() + ''
return text
def create_error_response( def create_error_response(
error_code: ErrorCode, error_code: ErrorCode,
message: str, message: str,
+2 -4
View File
@@ -8,9 +8,7 @@ is_odds_available's ML-blind truth table, the fixed format_odds_summary
gate (money-line-only odds now format), get_odds_for_games, and gate (money-line-only odds now format), get_odds_for_games, and
configuration loading. configuration loading.
No real network: requests.Session.get is always patched. The odds path sends No real network: src.base_odds_manager.requests.get is always patched.
its requests through a session so it can identify itself to ESPN, so patching
the module-level requests.get would no longer intercept anything.
""" """
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
@@ -61,7 +59,7 @@ def manager(cache_manager):
@pytest.fixture @pytest.fixture
def mock_get(): def mock_get():
with patch('src.base_odds_manager.requests.Session.get') as m: with patch('src.base_odds_manager.requests.get') as m:
m.return_value = _make_response({'items': [dict(FULL_ITEM)]}) m.return_value = _make_response({'items': [dict(FULL_ITEM)]})
yield m yield m
-146
View File
@@ -1,146 +0,0 @@
"""Tests that one cache directory gets one cleanup thread per process.
The sweep lists a directory and deletes from it, so a second thread over the
same directory only duplicates the scan. Nothing enforced that: every
CacheManager started its own, and since the loop closes over `self`, a
discarded manager could never be collected -- its thread stayed alive and
re-scanned the same directory every 24 hours for the life of the process.
On the dev rig a display process carried three, for one cache directory:
14:22:59.954 display_controller (the real one)
14:22:59.973 startup validation, run 1 (discarded)
14:23:01.055 startup validation, run 2 (discarded)
Startup validation runs twice and built a throwaway manager each time, purely
to read a directory path.
"""
import threading
import pytest
from src.cache_manager import CacheManager
@pytest.fixture(autouse=True)
def _clean_registry():
CacheManager._cleanup_owners.clear()
yield
for owner in list(CacheManager._cleanup_owners.values()):
owner.stop_cleanup_thread()
CacheManager._cleanup_owners.clear()
def _live_cleanup_threads():
return [t for t in threading.enumerate()
if t.name == 'DiskCacheCleanup' and t.is_alive()]
@pytest.fixture
def manager(tmp_path, monkeypatch):
"""A CacheManager pinned to a temp dir, so tests never touch the real one."""
monkeypatch.setattr(CacheManager, '_get_writable_cache_dir',
lambda self: str(tmp_path))
return CacheManager
class TestOneThreadPerDirectory:
def test_a_single_manager_starts_one(self, manager):
before = len(_live_cleanup_threads())
m = manager()
try:
assert len(_live_cleanup_threads()) == before + 1
finally:
m.stop_cleanup_thread()
def test_three_managers_still_start_one(self, manager):
# Exactly the rig's shape: the real manager plus two throwaways.
before = len(_live_cleanup_threads())
managers = [manager() for _ in range(3)]
try:
assert len(_live_cleanup_threads()) == before + 1
finally:
for m in managers:
m.stop_cleanup_thread()
def test_the_first_one_owns_it(self, manager):
first, second = manager(), manager()
try:
assert CacheManager._cleanup_owners[first.cache_dir] is first
assert second._cleanup_thread is None
finally:
first.stop_cleanup_thread()
second.stop_cleanup_thread()
def test_the_survivor_can_take_over(self, manager):
first = manager()
first.stop_cleanup_thread()
assert not _live_cleanup_threads()
second = manager()
try:
# Ownership was released, so the directory is swept again rather
# than being left permanently unclaimed by a dead owner.
assert len(_live_cleanup_threads()) == 1
assert CacheManager._cleanup_owners[second.cache_dir] is second
finally:
second.stop_cleanup_thread()
def test_stopping_a_non_owner_does_not_unclaim_the_directory(self, manager):
first, second = manager(), manager()
try:
second.stop_cleanup_thread() # never owned it
assert CacheManager._cleanup_owners[first.cache_dir] is first
assert len(_live_cleanup_threads()) == 1
finally:
first.stop_cleanup_thread()
def test_separate_directories_get_separate_threads(self, tmp_path, monkeypatch):
a, b = tmp_path / 'a', tmp_path / 'b'
a.mkdir()
b.mkdir()
dirs = iter([str(a), str(b)])
monkeypatch.setattr(CacheManager, '_get_writable_cache_dir',
lambda self: next(dirs))
first, second = CacheManager(), CacheManager()
try:
assert first.cache_dir != second.cache_dir
assert len(_live_cleanup_threads()) == 2
finally:
first.stop_cleanup_thread()
second.stop_cleanup_thread()
def test_no_thread_leaks_across_many_constructions(self, manager):
before = len(_live_cleanup_threads())
made = [manager() for _ in range(12)]
try:
assert len(_live_cleanup_threads()) == before + 1
finally:
for m in made:
m.stop_cleanup_thread()
assert len(_live_cleanup_threads()) == before
class TestValidatorDoesNotBuildItsOwn:
def test_it_uses_the_cache_manager_it_is_given(self, manager):
from src.startup_validator import StartupValidator
shared = manager()
try:
before = len(_live_cleanup_threads())
v = StartupValidator(config_manager=object(), cache_manager=shared)
v._validate_cache_directory()
assert len(_live_cleanup_threads()) == before, (
"validation started another cleanup thread")
finally:
shared.stop_cleanup_thread()
def test_without_one_it_cleans_up_after_itself(self, manager):
from src.startup_validator import StartupValidator
before = len(_live_cleanup_threads())
v = StartupValidator(config_manager=object())
v._validate_cache_directory()
assert len(_live_cleanup_threads()) == before, (
"the fallback manager left its cleanup thread running")
-188
View File
@@ -1,188 +0,0 @@
"""Tests that abandoned cache temp files get collected.
DiskCache.set() writes through tempfile.mkstemp and os.replace, removing its
own temp file in a finally. That covers a failed write, but not a process that
dies between the two -- a SIGKILL, a lost restart race, a power cut, all
ordinary on a Pi. Nothing collected what was left behind: the temp names are
".<key>.json.<random>", and the expiry sweep only listed names ending in
.json, so they accumulated for as long as the card had been in service.
Measured on a live rig before this fix: 76 orphans totalling 1,050 MB -- 81%
of the entire cache directory -- the oldest six months old.
The predicate that decides what to delete is tested harder than the sweep
itself, because a false positive here destroys real data.
"""
import os
import time
import pytest
from src.cache.disk_cache import DiskCache, _ORPHAN_TEMP_MAX_AGE_SECONDS
class FakeStrategy:
@staticmethod
def get_data_type_from_key(key):
return 'default'
POLICIES = {'default': 30}
@pytest.fixture
def cache(tmp_path):
return DiskCache(str(tmp_path))
def _age(path, seconds):
old = time.time() - seconds
os.utime(path, (old, old))
def _write(tmp_path, name, body='{}'):
p = tmp_path / name
p.write_text(body, encoding='utf-8')
return p
class TestWhatCountsAsAnOrphan:
@pytest.mark.parametrize('name', [
'.weather.json.a1b2c3d4',
'.odds_espn_football_nfl_401.json.xyz00000',
'.a.json.b',
])
def test_our_temp_files_are_orphans(self, name):
assert DiskCache._is_orphaned_temp(name)
@pytest.mark.parametrize('name', [
'weather.json', # real data
'.weather.json', # a dotted key that completed
'.gitignore', # not ours
'.hidden', # not ours
'weather.json.bak', # no leading dot: someone else's
'.json.abc', # no key between the dot and .json.
'.weather.json.', # no random component
'notes.txt',
])
def test_everything_else_is_left_alone(self, name):
assert not DiskCache._is_orphaned_temp(name)
def test_the_names_set_actually_creates_are_matched(self, cache, tmp_path):
"""Guard against the predicate and the writer drifting apart."""
created = []
real = os.replace
def capture(src, dst):
created.append(os.path.basename(src))
return real(src, dst)
import src.cache.disk_cache as mod
mod.os.replace = capture
try:
cache.set('weather', {'v': 1})
finally:
mod.os.replace = real
assert created, "set() did not go through the temp-file path"
assert all(DiskCache._is_orphaned_temp(n) for n in created), created
class TestTheSweep:
def test_an_old_orphan_is_removed(self, cache, tmp_path):
p = _write(tmp_path, '.weather.json.a1b2c3d4', 'x' * 5000)
_age(p, _ORPHAN_TEMP_MAX_AGE_SECONDS + 60)
stats = cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert not p.exists()
assert stats['orphan_temp_files_deleted'] == 1
assert stats['space_freed_bytes'] >= 5000
def test_an_in_flight_write_is_not_snatched_away(self, cache, tmp_path):
# The whole risk of this sweep: deleting a temp file another thread is
# about to os.replace into place.
p = _write(tmp_path, '.weather.json.inflight')
cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert p.exists()
def test_real_cache_files_survive(self, cache, tmp_path):
fresh = _write(tmp_path, 'weather.json')
dotted = _write(tmp_path, '.weather.json')
_age(dotted, _ORPHAN_TEMP_MAX_AGE_SECONDS + 60)
cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert fresh.exists()
assert dotted.exists(), "a completed .json was treated as a temp file"
def test_unrelated_dotfiles_survive(self, cache, tmp_path):
keep = _write(tmp_path, '.gitignore')
_age(keep, 400 * 86400)
cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert keep.exists()
def test_expiry_still_works_alongside_it(self, cache, tmp_path):
stale = _write(tmp_path, 'old.json')
_age(stale, 40 * 86400) # past the 30-day default
orphan = _write(tmp_path, '.old.json.zz999999')
_age(orphan, _ORPHAN_TEMP_MAX_AGE_SECONDS + 60)
stats = cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert not stale.exists()
assert not orphan.exists()
assert stats['files_deleted'] == 2
assert stats['orphan_temp_files_deleted'] == 1
def test_the_rig_scenario(self, cache, tmp_path):
"""76 orphans of assorted ages, none of them reachable before."""
for i in range(76):
p = _write(tmp_path, '.sched_%d.json.r%06d' % (i, i), 'x' * 1000)
_age(p, (i + 2) * 86400)
keep = _write(tmp_path, 'sched.json')
stats = cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert stats['orphan_temp_files_deleted'] == 76
assert keep.exists()
assert not list(tmp_path.glob('.sched_*'))
# The summary line is "<deleted>/<scanned>", so an orphan that is
# deleted but never counted as scanned renders as "76/1".
assert stats['files_scanned'] == 77
assert stats['files_deleted'] <= stats['files_scanned']
def test_deleted_never_exceeds_scanned(self, cache, tmp_path):
p = _write(tmp_path, '.only.json.a1b2c3d4')
_age(p, _ORPHAN_TEMP_MAX_AGE_SECONDS + 60)
stats = cache.cleanup_expired_files(FakeStrategy(), POLICIES)
assert stats['files_deleted'] == 1
assert stats['files_scanned'] == 1
def test_a_missing_file_mid_sweep_is_not_an_error(self, cache, tmp_path):
p = _write(tmp_path, '.weather.json.a1b2c3d4')
_age(p, _ORPHAN_TEMP_MAX_AGE_SECONDS + 60)
import src.cache.disk_cache as mod
real = mod.os.path.getsize
def vanish(path):
if path.endswith('.a1b2c3d4'):
os.remove(path)
raise FileNotFoundError(path)
return real(path)
mod.os.path.getsize = vanish
try:
stats = cache.cleanup_expired_files(FakeStrategy(), POLICIES)
finally:
mod.os.path.getsize = real
assert stats['errors'] == 0
-130
View File
@@ -1,130 +0,0 @@
"""Tests that a per-entry ttl actually controls expiry.
Regression under test: `CacheManager.set(key, data, ttl=...)` stored the value
and no read path ever consulted it. Expiry came from a `max_age` inferred from
substrings in the key ("live", "odds", "stock"), so every caller passing `ttl=`
-- 48 sites across the plugins and 4 in the core -- was writing a number that
did nothing. The old docstring admitted as much: "stored for compatibility but
expiration is still controlled via max_age when reading".
Measured against a real device's cache (8,873 entries carrying a ttl), the
inferred value and the intended one disagreed almost everywhere:
stocks max_age 600 vs ttl 1800 4903 entries
news max_age 3600 vs ttl 600 1770 entries
odds max_age 1800 vs ttl 3600 1301 entries
images max_age 300 vs ttl 2592000 20 entries
No `sports_live` entry carries a ttl, so live scores keep their inferred
30-second freshness either way.
"""
import time
import pytest
from src.cache.memory_cache import MemoryCache
from src.cache.disk_cache import DiskCache
@pytest.fixture
def disk(tmp_path):
return DiskCache(cache_dir=str(tmp_path))
def _record(ttl=None, age=0.0):
rec = {"data": {"v": 1}, "timestamp": time.time() - age}
if ttl is not None:
rec["ttl"] = ttl
return rec
class TestDiskCacheHonoursTtl:
def test_ttl_longer_than_max_age_keeps_the_entry(self, disk):
# The odds case: written wanting an hour, expired at 30 minutes.
disk.set("odds_espn_football_nfl_401", _record(ttl=3600, age=1900))
assert disk.get("odds_espn_football_nfl_401", max_age=1800) is not None
def test_ttl_shorter_than_max_age_expires_the_entry(self, disk):
# The news case: written wanting 10 minutes, kept for an hour.
disk.set("news_NHL_1", _record(ttl=600, age=900))
assert disk.get("news_NHL_1", max_age=3600) is None
def test_without_a_ttl_max_age_still_applies(self, disk):
disk.set("plain_key", _record(age=400))
assert disk.get("plain_key", max_age=300) is None
disk.set("plain_key2", _record(age=100))
assert disk.get("plain_key2", max_age=300) is not None
def test_a_fresh_entry_within_its_ttl_survives(self, disk):
disk.set("k", _record(ttl=600, age=10))
assert disk.get("k", max_age=30) is not None
def test_ttl_zero_expires_immediately(self, disk):
# 0 means zero seconds, not "forever" -- max_age=None is how a caller
# asks for no expiry.
disk.set("k", _record(ttl=0, age=1))
assert disk.get("k", max_age=99999) is None
@pytest.mark.parametrize("bad", ["600", None, True, False, -5, {"a": 1}])
def test_a_nonsense_ttl_falls_back_to_max_age(self, disk, bad):
# Including bools: True is an int in Python and must not become a 1s ttl.
rec = _record(age=400)
rec["ttl"] = bad
disk.set("k_%s" % type(bad).__name__, rec)
assert disk.get("k_%s" % type(bad).__name__, max_age=300) is None
class TestMemoryCacheHonoursTtl:
def test_ttl_longer_than_max_age_keeps_the_entry(self):
m = MemoryCache()
m.set("k", _record(ttl=3600))
m._timestamps["k"] = time.time() - 1900
assert m.get("k", max_age=1800) is not None
def test_ttl_shorter_than_max_age_expires_the_entry(self):
m = MemoryCache()
m.set("k", _record(ttl=600))
m._timestamps["k"] = time.time() - 900
assert m.get("k", max_age=3600) is None
def test_without_a_ttl_max_age_still_applies(self):
m = MemoryCache()
m.set("k", _record())
m._timestamps["k"] = time.time() - 400
assert m.get("k", max_age=300) is None
def test_both_layers_agree(self, tmp_path):
"""A record must not be live in one layer and expired in the other."""
rec = _record(ttl=3600, age=1900)
d = DiskCache(cache_dir=str(tmp_path))
d.set("k", rec)
m = MemoryCache()
m.set("k", rec)
m._timestamps["k"] = rec["timestamp"]
assert (d.get("k", max_age=1800) is not None) == (m.get("k", max_age=1800) is not None)
class TestEndToEnd:
def test_set_then_get_respects_the_ttl(self, tmp_path, monkeypatch):
"""The behaviour a caller of CacheManager.set(ttl=...) expects."""
from src.cache_manager import CacheManager
cm = CacheManager()
cm._disk_cache_component = DiskCache(cache_dir=str(tmp_path))
cm._memory_cache_component = MemoryCache()
cm.set("odds_espn_football_nfl_401", {"spread": 6.5}, ttl=3600)
# Age the stored record past the inferred max_age for odds (1800s) but
# within the ttl the caller asked for.
path = cm._disk_cache_component.get_cache_path("odds_espn_football_nfl_401")
import json
rec = json.load(open(path))
rec["timestamp"] = time.time() - 1900
json.dump(rec, open(path, "w"))
cm._memory_cache_component.clear() if hasattr(
cm._memory_cache_component, "clear") else None
got = cm.get_with_auto_strategy("odds_espn_football_nfl_401")
assert got is not None, "the ttl the caller asked for was ignored"
-42
View File
@@ -237,45 +237,3 @@ class TestDisplayManagerDoubleSided:
suppress_test_pattern=True) suppress_test_pattern=True)
assert dm.set_brightness(70) is True assert dm.set_brightness(70) is True
assert mock_rgb_matrix['matrix_instance'].brightness == 70 assert mock_rgb_matrix['matrix_instance'].brightness == 70
class TestDisplayManagerOrientation:
"""The orientation setting composes onto pixel_mapper_config for panels
mounted upside down, without disturbing a custom pixel_mapper_config."""
def _config(self, **hardware_overrides):
config = {
'display': {
'hardware': {
'rows': 32, 'cols': 64, 'chain_length': 2, 'parallel': 1,
'hardware_mapping': 'adafruit-hat-pwm', 'brightness': 90,
},
'runtime': {'gpio_slowdown': 2},
},
'timezone': 'UTC',
'plugin_system': {'plugins_directory': 'plugins'},
}
config['display']['hardware'].update(hardware_overrides)
return config
def test_default_orientation_leaves_pixel_mapper_config_untouched(self, mock_rgb_matrix):
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'false'}):
DisplayManager(self._config(), suppress_test_pattern=True)
options = mock_rgb_matrix['options_class'].return_value
assert options.pixel_mapper_config == ''
def test_orientation_180_appends_rotate_mapper(self, mock_rgb_matrix):
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'false'}):
DisplayManager(self._config(orientation='180'), suppress_test_pattern=True)
options = mock_rgb_matrix['options_class'].return_value
assert options.pixel_mapper_config == 'Rotate:180'
def test_orientation_180_composes_with_existing_pixel_mapper_config(self, mock_rgb_matrix):
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'false'}):
DisplayManager(self._config(orientation='180', pixel_mapper_config='U-mapper'),
suppress_test_pattern=True)
options = mock_rgb_matrix['options_class'].return_value
assert options.pixel_mapper_config == 'U-mapper;Rotate:180'
-224
View File
@@ -1,224 +0,0 @@
"""Tests that startup does not wait indefinitely for plugins to fetch data.
DisplayController.__init__ calls _update_modules() once, to populate plugin
data before the first frame. It walks every loaded plugin in turn, and each
update blocks the calling thread for up to the executor's 30s timeout, so the
uncapped total is the sum of every slow plugin on the system.
Profiled on a live rig with py-spy, the main thread sat 9.34s in
display_controller._update_modules
-> plugin_executor.execute_update
-> execute_with_timeout -> threading.join
and the controller's own log put the full pass at 82 seconds on the worst
boot measured (55 and 26 on the two before). The panel shows nothing for all
of it.
Nothing is lost by stopping early: a plugin that has never updated is
immediately due, so run_scheduled_updates() collects it seconds later with the
display already running.
"""
import os
import time
from unittest.mock import Mock
import pytest
# display_controller imports display_manager, which binds the hardware
# rgbmatrix module unless EMULATOR=true is set before import (same convention
# as test_display_controller_vegas_tick.py).
os.environ.setdefault("EMULATOR", "true")
from src.display_controller import ( # noqa: E402
DisplayController, _INITIAL_UPDATE_BUDGET_SECONDS,
_MIN_INITIAL_UPDATE_TIMEOUT_SECONDS)
class FakeExecutor:
"""Records which plugins were updated, and can make some of them slow."""
def __init__(self, cost=0.0, slow=()):
self.updated = []
self.cost = cost
self.slow = set(slow)
def execute_update(self, plugin, plugin_id, timeout=None):
self.updated.append(plugin_id)
if plugin_id in self.slow:
time.sleep(self.cost)
return True
@pytest.fixture
def tiny_floor(monkeypatch):
"""Shrink the "worth starting" floor so timing tests stay quick."""
import src.display_controller as mod
monkeypatch.setattr(mod, "_MIN_INITIAL_UPDATE_TIMEOUT_SECONDS", 0.01)
def _controller(plugin_ids, executor):
c = DisplayController.__new__(DisplayController)
c.plugin_manager = Mock()
# Both attributes, because _update_modules reads
# `loaded_plugins or plugins` and an empty dict is falsy.
c.plugin_manager.loaded_plugins = {pid: Mock() for pid in plugin_ids}
c.plugin_manager.plugins = dict(c.plugin_manager.loaded_plugins)
c.plugin_manager.plugin_executor = executor
c.plugin_manager.plugin_last_update = {}
c.plugin_manager.health_tracker = None
return c
class TestTheBudgetIsRespected:
def test_without_a_deadline_every_plugin_is_updated(self):
ex = FakeExecutor()
_controller(['a', 'b', 'c'], ex)._update_modules()
assert ex.updated == ['a', 'b', 'c']
def test_a_passed_deadline_stops_the_pass(self):
ex = FakeExecutor()
_controller(['a', 'b', 'c'], ex)._update_modules(deadline=time.time() - 1)
assert ex.updated == [], "updated %r after the deadline" % ex.updated
def test_slow_plugins_do_not_drag_in_the_rest(self, tiny_floor):
# One plugin burns the whole budget; the remainder must be left alone
# rather than each adding its own wait.
ex = FakeExecutor(cost=0.3, slow={'slow'})
c = _controller(['slow'] + ['p%d' % i for i in range(20)], ex)
started = time.time()
c._update_modules(deadline=started + 0.2)
elapsed = time.time() - started
assert ex.updated == ['slow'], "updated %r" % ex.updated
# Bounded by the one in-flight update, not by twenty more.
assert elapsed < 1.0, "%.2fs" % elapsed
def test_a_generous_deadline_still_gets_everything(self):
ex = FakeExecutor()
c = _controller(['a', 'b', 'c'], ex)
c._update_modules(deadline=time.time() + 30)
assert ex.updated == ['a', 'b', 'c']
def test_the_deadline_is_checked_before_each_plugin(self, tiny_floor):
# Not just once up front: the budget can be spent partway through.
ex = FakeExecutor(cost=0.15, slow={'a', 'b', 'c', 'd'})
c = _controller(['a', 'b', 'c', 'd'], ex)
c._update_modules(deadline=time.time() + 0.2)
assert 0 < len(ex.updated) < 4, "updated %r" % ex.updated
class TestThePassIsBoundedInPractice:
def test_the_last_plugin_cannot_overrun_the_budget(self):
# Checking the deadline before each plugin is not enough on its own:
# one that starts with a moment left could still block for the
# executor's full timeout. On the rig that turned a 20s budget into a
# 31.8s pass, so the remaining budget is passed down as the timeout.
seen = []
class Executor:
def execute_update(self, plugin, plugin_id, timeout=None):
seen.append(timeout)
return True
c = _controller(['a', 'b', 'c'], Executor())
deadline = time.time() + 5
c._update_modules(deadline=deadline)
assert seen and all(t is not None for t in seen), seen
assert all(t <= 5.01 for t in seen), seen
# The exact remainder, never clamped up: clamping would let the pass
# run past its deadline. Anything below the floor is deferred instead,
# so what does start always has a usable slot.
assert all(t >= _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS for t in seen), seen
def test_without_a_deadline_the_executor_default_is_left_alone(self):
seen = []
class Executor:
def execute_update(self, plugin, plugin_id, timeout=None):
seen.append(timeout)
return True
_controller(['a'], Executor())._update_modules()
assert seen == [None], seen
class TestTheBudgetItself:
def test_it_is_short_enough_to_be_worth_having(self):
# The measured uncapped worst case was 82s; a budget near that would
# not bound anything.
assert _INITIAL_UPDATE_BUDGET_SECONDS <= 30
def test_it_is_long_enough_for_a_quick_plugin_or_two(self):
assert _INITIAL_UPDATE_BUDGET_SECONDS >= 5
class TestNothingIsSilentlyDropped:
def test_deferred_plugins_are_named_in_the_log(self, caplog):
ex = FakeExecutor()
c = _controller(['a', 'b'], ex)
with caplog.at_level('INFO'):
c._update_modules(deadline=time.time() - 1)
text = "\n".join(r.getMessage() for r in caplog.records)
assert 'a' in text and 'b' in text, text
assert 'budget' in text.lower(), text
def test_nothing_is_logged_when_all_of_them_ran(self, caplog):
ex = FakeExecutor()
c = _controller(['a'], ex)
with caplog.at_level('INFO'):
c._update_modules(deadline=time.time() + 30)
assert not any('budget' in r.getMessage().lower() for r in caplog.records)
class TestItDoesNotBreakTheOrdinaryPaths:
def test_no_plugin_manager_is_harmless(self):
c = DisplayController.__new__(DisplayController)
c.plugin_manager = None
c._update_modules(deadline=time.time() - 1) # must not raise
def test_an_empty_plugin_set_is_harmless(self):
ex = FakeExecutor()
_controller([], ex)._update_modules(deadline=time.time() + 5)
assert ex.updated == []
class TestTooLittleBudgetDefersRatherThanClamps:
def test_a_plugin_starting_below_the_floor_is_deferred(self):
ex = FakeExecutor()
c = _controller(['a'], ex)
# Just under the floor: previously this was clamped up to the floor and
# run anyway, which pushed the pass past its deadline.
c._update_modules(
deadline=time.time() + _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS - 0.05)
assert ex.updated == [], "started a plugin it could not give a slot to"
def test_a_plugin_starting_above_the_floor_still_runs(self):
ex = FakeExecutor()
c = _controller(['a'], ex)
c._update_modules(
deadline=time.time() + _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS + 1)
assert ex.updated == ['a']
def test_the_timeout_is_the_remainder_not_the_floor(self):
seen = []
class Executor:
def execute_update(self, plugin, plugin_id, timeout=None):
seen.append(timeout)
return True
c = _controller(['a'], Executor())
c._update_modules(deadline=time.time() + 9)
assert seen and 8.5 <= seen[0] <= 9.01, seen
def test_the_pass_cannot_outlast_its_deadline(self, tiny_floor):
# Every plugin sleeps well past the budget; the deferral keeps the
# whole pass inside it rather than overrunning by a floor's worth.
ex = FakeExecutor(cost=0.4, slow={'a', 'b', 'c', 'd', 'e'})
c = _controller(['a', 'b', 'c', 'd', 'e'], ex)
started = time.time()
c._update_modules(deadline=started + 0.5)
assert time.time() - started < 1.2, "%.2fs" % (time.time() - started)
-190
View File
@@ -1,190 +0,0 @@
"""Tests the startup screen that shows while plugins fetch their first data.
That screen is on the panel for the whole initial-update window, and on a
headless Pi it is the only place the device's address appears without going
looking for it -- so it now carries the address as well as "Initializing".
Two things have to hold. It must fit every supported panel: the old fixed
8px PressStart2P drew "Initializing" 96px wide at x=10, which ran off the
side of a 64px panel before an address was ever added. And the lookup must be
cheap, because this runs on the startup path that the rest of this change
exists to shorten.
"""
import os
import time
from PIL import Image, ImageDraw, ImageFont
import pytest
os.environ.setdefault("EMULATOR", "true")
from src.display_manager import DisplayManager # noqa: E402
SIZES = [(64, 32), (128, 32), (128, 64), (256, 32), (512, 64)]
class FakeMatrix:
def __init__(self, width, height):
self.width, self.height = width, height
def _manager(width, height):
dm = DisplayManager.__new__(DisplayManager)
dm.image = Image.new('RGB', (width, height))
dm.draw = ImageDraw.Draw(dm.image)
dm.matrix = FakeMatrix(width, height)
dm.font = ImageFont.truetype('assets/fonts/PressStart2P-Regular.ttf', 8)
return dm
def _layout(dm, lines):
"""The geometry _draw_startup_banner uses."""
font = dm._fitting_font(lines, dm.matrix.width - 2)
line_height = dm.draw.textbbox((0, 0), "Ag", font=font)[3] + 1
top = max(1, (dm.matrix.height - line_height * len(lines)) // 2)
widths = [dm.draw.textlength(t, font=font) for t in lines]
return font, widths, top, top + line_height * len(lines)
def _render_over_pattern(width, height, lines):
"""Draw the test pattern, then the banner over it, as startup does."""
dm = _manager(width, height)
dm.draw.rectangle([0, 0, width - 1, height - 1], outline=(255, 0, 0))
dm.draw.line([0, 0, width - 1, height - 1], fill=(0, 255, 0))
dm._draw_startup_banner(lines, width, height)
return dm
class TestTheAddressLookup:
def test_it_never_reports_loopback(self):
# A loopback address on the panel would be actively misleading -- it is
# not something anyone can browse to.
ip = DisplayManager._local_ip()
assert ip is None or not ip.startswith("127."), ip
def test_it_looks_like_an_address_when_there_is_one(self):
ip = DisplayManager._local_ip()
if ip is None:
pytest.skip("host has no routable address")
parts = ip.split(".")
assert len(parts) == 4 and all(p.isdigit() for p in parts), ip
def test_it_is_cheap_enough_for_the_startup_path(self):
DisplayManager._local_ip() # warm anything cacheable
started = time.perf_counter()
for _ in range(20):
DisplayManager._local_ip()
per_call = (time.perf_counter() - started) / 20
# `hostname -I` with its 2s timeout, which the web launcher uses, would
# be thousands of times this.
assert per_call < 0.05, "%.1f ms per call" % (per_call * 1000)
def test_it_returns_none_rather_than_raising(self, monkeypatch):
import src.display_manager as mod
def no_network(*a, **k):
raise OSError("network is unreachable")
monkeypatch.setattr(mod.socket, "socket", no_network)
assert DisplayManager._local_ip() is None
class TestItFitsEveryPanel:
@pytest.mark.parametrize("width,height", SIZES)
def test_both_lines_fit_with_an_address(self, width, height):
dm = _manager(width, height)
_font, widths, top, bottom = _layout(dm, ["Initializing", "255.255.255.255"])
assert all(w <= width - 2 for w in widths), (width, widths)
assert bottom <= height and top >= 0, (top, bottom, height)
@pytest.mark.parametrize("width,height", SIZES)
def test_it_still_fits_with_no_address(self, width, height):
dm = _manager(width, height)
_font, widths, _top, bottom = _layout(dm, ["Initializing"])
assert all(w <= width - 2 for w in widths), (width, widths)
assert bottom <= height, (bottom, height)
def test_the_smallest_panel_drops_to_a_narrower_font(self):
# The regression this guards: PressStart2P at 8px is 96px wide for
# "Initializing", which does not fit 64px however it is positioned.
dm = _manager(64, 32)
font, widths, _t, _b = _layout(dm, ["Initializing", "10.0.20.104"])
assert font is not dm.font, "kept a font that cannot fit"
assert max(widths) <= 62, widths
def test_a_roomy_panel_keeps_the_larger_font(self):
dm = _manager(256, 32)
font, _w, _t, _b = _layout(dm, ["Initializing", "10.0.20.104"])
assert font is dm.font, "needlessly shrank on a panel with room"
class TestPlacement:
@pytest.mark.parametrize("width,height", SIZES)
def test_the_lines_are_centred(self, width, height):
dm = _manager(width, height)
lines = ["Initializing", "10.0.20.104"]
_font, widths, _t, _b = _layout(dm, lines)
for w in widths:
left = max(0, (width - w) // 2)
assert abs((left + (left + w)) - width) <= 2, (left, w, width)
def test_the_address_sits_under_the_word(self):
dm = _manager(128, 64)
font, _w, top, bottom = _layout(dm, ["Initializing", "10.0.20.104"])
line_height = dm.draw.textbbox((0, 0), "Ag", font=font)[3] + 1
assert bottom - top == line_height * 2
class TestItIsActuallyReadable:
"""The point of the address is that someone can read it off the wall."""
@pytest.mark.parametrize("width,height", SIZES)
def test_the_diagonal_does_not_cross_the_text(self, width, height):
lines = ["Initializing", "10.0.20.104"]
dm = _render_over_pattern(width, height, lines)
_font, widths, top, bottom = _layout(dm, lines)
# textlength returns a float, so these must be floored before they
# can index pixels.
block_width = int(max(widths))
left = int(max(0, (width - block_width) // 2))
px = dm.image.load()
green = 0
for y in range(int(top), min(int(bottom), height)):
for x in range(left, min(left + block_width, width)):
r, g, b = px[x, y]
if g > 128 and r < 128 and b < 128:
green += 1
assert green == 0, "%d green pixels behind the text at %dx%d" % (
green, width, height)
@pytest.mark.parametrize("width,height", SIZES)
def test_the_text_stays_pure_blue(self, width, height):
# Not a style choice. The pattern lights one channel per element --
# red border, green diagonal, blue text -- so a glance says whether
# led_rgb_sequence is right: wire it BGR and the border comes up blue
# and this text red. White text would light all three and destroy the
# only blue reference on the screen.
dm = _render_over_pattern(width, height, ["Initializing", "10.0.20.104"])
px = dm.image.load()
blue = sum(1 for y in range(height) for x in range(width)
if px[x, y] == (0, 0, 255))
assert blue > 20, "only %d blue pixels at %dx%d" % (blue, width, height)
white = sum(1 for y in range(height) for x in range(width)
if px[x, y] == (255, 255, 255))
assert white == 0, "%d white pixels would muddy the channel check" % white
def test_each_element_lights_one_channel(self):
# The whole point of the pattern: three pure primaries on screen.
dm = _render_over_pattern(128, 64, ["Initializing", "10.0.20.104"])
seen = set(dm.image.getdata())
assert (255, 0, 0) in seen, "no pure red border"
assert (0, 255, 0) in seen, "no pure green diagonal"
assert (0, 0, 255) in seen, "no pure blue text"
def test_nothing_is_drawn_for_no_lines(self):
dm = _manager(128, 64)
before = dm.image.tobytes()
dm._draw_startup_banner([], 128, 64)
assert dm.image.tobytes() == before
+44 -84
View File
@@ -10,15 +10,10 @@ and the update carrying every game's score was killed:
Invisible out of season -- preseason week 1 returns a single game -- and a Invisible out of season -- preseason week 1 returns a single game -- and a
Sunday slate is around sixteen. Sunday slate is around sixteen.
The request now goes through a session that identifies the caller, so the
tests patch `manager.session.get` rather than the module's `requests.get`.
""" """
from unittest.mock import Mock from unittest.mock import Mock
import requests
from src.base_odds_manager import BaseOddsManager from src.base_odds_manager import BaseOddsManager
PLUGIN_BUDGET = 30.0 # PluginExecutor(default_timeout=30.0) PLUGIN_BUDGET = 30.0 # PluginExecutor(default_timeout=30.0)
@@ -30,83 +25,43 @@ def _manager(cache=None):
return BaseOddsManager(cache_manager=cache, config_manager=None) return BaseOddsManager(cache_manager=cache, config_manager=None)
def _timing_out(manager):
"""Point the manager's session at a request that always times out."""
manager.session.get = Mock(side_effect=requests.exceptions.Timeout("x"))
return manager.session.get
def _returning(manager, payload):
resp = Mock()
resp.json.return_value = payload
resp.raise_for_status.return_value = None
manager.session.get = Mock(return_value=resp)
return manager.session.get
class TestRequestTimeout: class TestRequestTimeout:
def test_leaves_room_in_the_operation_budget(self): def test_leaves_room_in_the_operation_budget(self):
assert _manager().request_timeout < PLUGIN_BUDGET / 2 assert _manager().request_timeout < PLUGIN_BUDGET / 2
def test_the_timeout_is_the_one_actually_used(self): def test_the_timeout_is_the_one_actually_used(self):
m = _manager() m = _manager()
get = _timing_out(m) import src.base_odds_manager as mod
real = mod.requests.get
try:
mod.requests.get = Mock(side_effect=mod.requests.exceptions.Timeout("x"))
m.get_odds("football", "nfl", "401") m.get_odds("football", "nfl", "401")
assert get.call_args.kwargs["timeout"] == m.request_timeout assert mod.requests.get.call_args.kwargs["timeout"] == m.request_timeout
finally:
mod.requests.get = real
class TestIdentifiesItselfToEspn:
"""ESPN 403s python-requests' default agent, and bare custom tokens.
What it accepts is a token carrying a URL that says who is calling. This
path used a bare requests.get and so sent the default -- the one thing
known to be rejected. Everything else in the tree that talks to ESPN
already sends the header below.
"""
def test_the_user_agent_names_the_project_and_links_to_it(self):
ua = _manager().session.headers["User-Agent"]
assert "python-requests" not in ua
assert "LEDMatrix" in ua
assert "github.com/ChuckBuilds/LEDMatrix" in ua
def test_it_is_the_same_agent_the_rest_of_the_tree_sends(self):
# Compared against the live value rather than a copied literal, so the
# two cannot drift apart the next time ESPN moves the goalposts.
from src.common.api_helper import APIHelper
assert (_manager().session.headers["User-Agent"]
== APIHelper().session.headers["User-Agent"])
def test_the_header_reaches_the_request(self):
m = _manager()
get = _returning(m, {})
m._extract_espn_data = Mock(return_value=None)
m.get_odds("football", "nfl", "401")
# Sent via the session, so it applies without being passed per-call.
assert get.call_count == 1
assert "User-Agent" in m.session.headers
def test_no_retry_adapter_multiplies_the_timeout(self):
# api_helper mounts a retrying adapter; this path must not, or a 5s
# timeout becomes 15s and the budget fix is undone.
m = _manager()
for adapter in m.session.adapters.values():
retries = getattr(adapter, "max_retries", None)
assert getattr(retries, "total", 0) in (0, None), (
"odds session mounts a retrying adapter (total=%r); retries "
"multiply request_timeout" % getattr(retries, "total", None))
class TestSlowEspnCannotKillTheUpdate: class TestSlowEspnCannotKillTheUpdate:
def test_one_failure_stops_the_rest_of_the_slate_hitting_the_network(self): def test_one_failure_stops_the_rest_of_the_slate_hitting_the_network(self):
m = _manager() m = _manager()
get = _timing_out(m) import src.base_odds_manager as mod
real = mod.requests.get
calls = {"n": 0}
def timeout(*a, **k):
calls["n"] += 1
raise mod.requests.exceptions.Timeout("timed out")
try:
mod.requests.get = timeout
for i in range(16): # a full slate, one game at a time for i in range(16): # a full slate, one game at a time
m.get_odds("football", "nfl", "4018730%02d" % i) m.get_odds("football", "nfl", "4018730%02d" % i)
finally:
mod.requests.get = real
assert get.call_count == 1, ( assert calls["n"] == 1, (
"%d games each paid the timeout; the breaker should have stopped " "%d games each paid the timeout; the breaker should have stopped "
"after the first" % get.call_count) "after the first" % calls["n"])
def test_worst_case_slate_stays_inside_the_budget(self): def test_worst_case_slate_stays_inside_the_budget(self):
m = _manager() m = _manager()
@@ -115,42 +70,41 @@ class TestSlowEspnCannotKillTheUpdate:
def test_recovery_is_automatic(self): def test_recovery_is_automatic(self):
m = _manager() m = _manager()
import src.base_odds_manager as mod import src.base_odds_manager as mod
real_monotonic = mod.time.monotonic real_get, real_monotonic = mod.requests.get, mod.time.monotonic
clock = {"t": 1000.0} clock = {"t": 1000.0}
try: try:
mod.time.monotonic = lambda: clock["t"] mod.time.monotonic = lambda: clock["t"]
get = _timing_out(m) mod.requests.get = Mock(
side_effect=mod.requests.exceptions.Timeout("timed out"))
m.get_odds("football", "nfl", "401") m.get_odds("football", "nfl", "401")
assert m._skip_network_until > clock["t"], "breaker did not open" assert m._skip_network_until > clock["t"], "breaker did not open"
clock["t"] += 1 clock["t"] += 1
before = get.call_count before = mod.requests.get.call_count
m.get_odds("football", "nfl", "402") m.get_odds("football", "nfl", "402")
assert get.call_count == before, "should not have retried" assert mod.requests.get.call_count == before, "should not have retried"
clock["t"] += m._FAILURE_COOLDOWN clock["t"] += m._FAILURE_COOLDOWN
m.get_odds("football", "nfl", "403") m.get_odds("football", "nfl", "403")
assert get.call_count > before, "never retried" assert mod.requests.get.call_count > before, "never retried"
finally: finally:
mod.time.monotonic = real_monotonic mod.requests.get, mod.time.monotonic = real_get, real_monotonic
def test_a_healthy_fetch_clears_the_breaker(self): def test_a_healthy_fetch_clears_the_breaker(self):
m = _manager() m = _manager()
m._skip_network_until = 0.0 m._skip_network_until = 0.0
m._extract_espn_data = Mock(return_value=None) m._extract_espn_data = Mock(return_value=None)
_returning(m, {}) import src.base_odds_manager as mod
m.get_odds("football", "nfl", "401") real = mod.requests.get
assert m._skip_network_until == 0.0 try:
def test_a_403_opens_the_breaker_rather_than_hammering(self):
# raise_for_status raises HTTPError, a RequestException -- so a wrong
# or missing agent backs off instead of 403ing once per game.
m = _manager()
resp = Mock() resp = Mock()
resp.raise_for_status.side_effect = requests.exceptions.HTTPError("403") resp.json.return_value = {}
m.session.get = Mock(return_value=resp) resp.raise_for_status.return_value = None
mod.requests.get = Mock(return_value=resp)
m.get_odds("football", "nfl", "401") m.get_odds("football", "nfl", "401")
assert m._skip_network_until > 0.0 finally:
mod.requests.get = real
assert m._skip_network_until == 0.0
def test_the_stale_cache_fallback_still_works(self): def test_the_stale_cache_fallback_still_works(self):
# The failing request must still hand back whatever was cached; only # The failing request must still hand back whatever was cached; only
@@ -158,5 +112,11 @@ class TestSlowEspnCannotKillTheUpdate:
cache = Mock() cache = Mock()
cache.get_with_auto_strategy.side_effect = [None, {"details": "stale"}] cache.get_with_auto_strategy.side_effect = [None, {"details": "stale"}]
m = BaseOddsManager(cache_manager=cache, config_manager=None) m = BaseOddsManager(cache_manager=cache, config_manager=None)
_timing_out(m) import src.base_odds_manager as mod
real = mod.requests.get
try:
mod.requests.get = Mock(
side_effect=mod.requests.exceptions.Timeout("timed out"))
assert m.get_odds("football", "nfl", "401") == {"details": "stale"} assert m.get_odds("football", "nfl", "401") == {"details": "stale"}
finally:
mod.requests.get = real
-241
View File
@@ -1,241 +0,0 @@
"""
Getting Started checklist: what the server decides, and what it must not.
The timezone step used to tick server-side when the saved timezone differed
from the shipped default, OR-ed with the saved city. That made the step
unsatisfiable for anyone genuinely in the default zone (the card nagged
forever), and let a saved city tick it off while the timezone was still wrong.
The step is now verified in the browser against its own zone, so the server's
only job is to hand over the configured value and stay out of the decision.
These tests pin that contract: the panel-size step still reflects config, the
timezone step never pre-ticks, it carries the configured zone, and the city
has no influence on it.
"""
import copy
import re
import sys
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from flask import Flask
PROJECT_ROOT = Path(__file__).parent.parent
sys.path.insert(0, str(PROJECT_ROOT))
BASE_CONFIG = {
"timezone": "America/New_York",
"location": {"city": "Tampa", "state": "Florida", "country": "US"},
"display": {
"hardware": {"rows": 32, "cols": 64, "chain_length": 2, "parallel": 1},
"runtime": {},
"double_sided": {"enabled": False},
"vegas_scroll": {"plugin_order": [], "excluded_plugins": []},
"plugin_rotation_order": [],
},
"plugin_system": {},
"schedule": {},
"dim_schedule": {},
"sync": {},
}
def render(config):
"""Render the overview partial against one config, as app.py would."""
base = PROJECT_ROOT / "web_interface"
app = Flask(
__name__,
template_folder=str(base / "templates"),
static_folder=str(base / "static"),
)
app.config["TESTING"] = True
from web_interface.blueprints import pages_v3 as pv
# pages_v3 is a module-level singleton shared across the test process;
# restore whatever the previous test left on it.
original_cm = getattr(pv.pages_v3, "config_manager", None)
original_pm = getattr(pv.pages_v3, "plugin_manager", None)
mock_cm = MagicMock()
mock_cm.load_config.return_value = config
mock_cm.get_raw_file_content.return_value = config
pv.pages_v3.config_manager = mock_cm
mock_pm = MagicMock()
mock_pm.plugins = {}
mock_pm.get_all_plugin_info.return_value = []
mock_pm.get_plugin_display_modes.side_effect = lambda pid: []
pv.pages_v3.plugin_manager = mock_pm
app.register_blueprint(pv.pages_v3, url_prefix="")
try:
resp = app.test_client().get("/partials/overview")
assert resp.status_code == 200, resp.status_code
return resp.get_data(as_text=True)
finally:
pv.pages_v3.config_manager = original_cm
pv.pages_v3.plugin_manager = original_pm
def timezone_step(body):
"""The checklist <button> for the timezone step."""
match = re.search(r"<button[^>]*data-check=\"timezone\"[^>]*>", body)
assert match, "timezone step not found in the rendered checklist"
return match.group(0)
def config_with(**overrides):
config = copy.deepcopy(BASE_CONFIG)
for key, value in overrides.items():
config[key] = value
return config
@pytest.mark.parametrize(
"timezone",
["America/New_York", "America/Los_Angeles", "Europe/Madrid", "Asia/Kolkata"],
)
def test_timezone_step_never_pre_ticks_server_side(timezone):
"""The browser owns this decision; the server must not pre-empt it.
The default zone is in the list deliberately: that is the case the old
default-comparison could never tick.
"""
step = timezone_step(render(config_with(timezone=timezone)))
assert 'data-done="0"' in step, step
@pytest.mark.parametrize(
"timezone",
["America/New_York", "Europe/Madrid", "Pacific/Auckland"],
)
def test_timezone_step_carries_the_configured_zone(timezone):
"""JS compares data-tz against the browser, so it has to be the real value."""
assert f'data-tz="{timezone}"' in timezone_step(render(config_with(timezone=timezone)))
def test_city_does_not_influence_the_timezone_step():
"""The coupling this change removes: city said nothing about the timezone,
and OR-ing it let a saved city tick the step off with the zone still wrong.
timezone_step() returns the opening tag only, so this compares the state
the step is in -- data-done and data-tz -- and not the label, which does
still show the configured city as context and so differs between the two.
"""
tampa = timezone_step(render(config_with(
location={"city": "Tampa", "state": "Florida", "country": "US"})))
seattle = timezone_step(render(config_with(
location={"city": "Seattle", "state": "Washington", "country": "US"})))
assert tampa == seattle
def test_missing_timezone_leaves_the_step_open():
"""Nothing saved means nothing to verify: the step stays unticked and the
JS bails on the empty value rather than comparing against ''."""
step = timezone_step(render(config_with(timezone="")))
assert 'data-tz=""' in step
assert 'data-done="0"' in step
def test_zone_comparison_asks_for_the_time_of_day():
"""Guard on the Intl options, which look like a stylistic choice.
dateStyle/timeStyle are late additions (Firefox shipped them in 91). An
implementation that does not know them ignores them and formats the date
alone -- which compares New York, Chicago and Madrid as equal and ticks
the step for a timezone that is plainly wrong. Explicit numeric fields
have been in Intl since ECMA-402 v1.
"""
template = (PROJECT_ROOT / "web_interface" / "templates" / "v3"
/ "partials" / "overview.html").read_text()
body = template[template.index("function sameZone"):]
body = body[:body.index("}())")]
# The comment above the options names dateStyle/timeStyle to explain why
# they are not used, so match on code only.
body = "\n".join(line for line in body.splitlines()
if not line.lstrip().startswith("//"))
assert "dateStyle" not in body and "timeStyle" not in body, (
"zone comparison must not depend on dateStyle/timeStyle")
for field in ("hour:", "minute:", "year:", "month:", "day:"):
assert field in body, f"zone comparison dropped {field!r}"
def test_zone_comparison_samples_both_sides_of_dst():
"""One instant is not enough, and the shortfall is invisible for months.
America/New_York and America/Lima hold the same offset all winter, so a
check against now alone ticks the step in January for a panel that runs an
hour off from March. The comparison has to sample instants either side of
DST -- mid-January and mid-July, which covers both hemispheres.
"""
template = (PROJECT_ROOT / "web_interface" / "templates" / "v3"
/ "partials" / "overview.html").read_text()
body = template[template.index("function sameZone"):]
body = body[:body.index("}())")]
code = "\n".join(line for line in body.splitlines()
if not line.lstrip().startswith("//"))
assert "Date.UTC" in code, (
"zone comparison samples only the current instant, so zones that "
"coincide seasonally would read as equal")
assert code.count("Date.UTC") >= 2, "expected an instant either side of DST"
def _stamp(zone, instant):
"""The JS comparison's algorithm, for pinning what it must decide.
There is no JS runtime here (and the repo has no JS test infra), so this
mirrors sameZone rather than executing it: same instants, same wall-clock
equality. It records the verdicts the shipped code has to reach.
"""
from zoneinfo import ZoneInfo
return instant.astimezone(ZoneInfo(zone)).strftime("%m/%d/%Y %H:%M")
@pytest.mark.parametrize(
"left,right,equivalent",
[
# Aliases: one zone under two names.
("Asia/Calcutta", "Asia/Kolkata", True),
("Europe/Kiev", "Europe/Kyiv", True),
# Same rules year-round: either renders the same times, so a panel set
# to one and browsed from the other is correctly configured.
("America/New_York", "America/Toronto", True),
# Coincide in winter only -- the case a single-instant check gets wrong.
("America/New_York", "America/Lima", False),
("America/Phoenix", "America/Los_Angeles", False),
("Australia/Sydney", "Pacific/Guadalcanal", False),
# Plainly different.
("America/New_York", "America/Chicago", False),
("America/New_York", "Europe/Madrid", False),
],
)
def test_which_zone_pairs_must_count_as_the_same(left, right, equivalent):
from datetime import datetime
from zoneinfo import ZoneInfo
year = 2026
instants = [datetime(year, 1, 15, 12, tzinfo=ZoneInfo("UTC")),
datetime(year, 7, 15, 12, tzinfo=ZoneInfo("UTC"))]
matched = all(_stamp(left, at) == _stamp(right, at) for at in instants)
assert matched is equivalent, (
f"{left} vs {right}: sampling both seasons gave {matched}")
@pytest.mark.parametrize(
"hardware,expected",
[
({"rows": 32, "cols": 64, "chain_length": 2, "parallel": 1}, "1"),
({"rows": 0, "cols": 0, "chain_length": 0, "parallel": 1}, "0"),
],
)
def test_panel_size_step_still_reflects_config(hardware, expected):
"""Regression guard: the hardware step is still decided server-side."""
config = config_with()
config["display"]["hardware"] = hardware
body = render(config)
match = re.search(r"<button[^>]*data-tab=\"display\"[^>]*>", body)
assert match, "panel-size step not found"
assert f'data-done="{expected}"' in match.group(0), match.group(0)
-64
View File
@@ -1,64 +0,0 @@
"""Tests the percentile used by the Vegas frame-time log line.
The FPS line reports p99 next to the worst frame, and the point of having both
is that they say different things: p99 is the bad-but-ordinary frame, worst is
the outlier. The obvious index, int(n * 0.99), is off by one and at exactly
100 samples selects the maximum -- so the two columns would report the same
number precisely when the sample was smallest.
"""
import math
import pytest
from src.vegas_mode.coordinator import _percentile
class TestNearestRank:
def test_a_hundred_samples_do_not_return_the_maximum(self):
ordered = [float(i) for i in range(100)] # 0..99
assert _percentile(ordered, 0.99) == 98.0
assert _percentile(ordered, 0.99) != max(ordered)
def test_it_matches_the_nearest_rank_definition(self):
for n in (1, 2, 3, 10, 99, 100, 101, 600, 1000):
ordered = [float(i) for i in range(n)]
expected = ordered[min(n - 1, max(0, math.ceil(n * 0.99) - 1))]
assert _percentile(ordered, 0.99) == expected, n
@pytest.mark.parametrize('fraction,expected', [
(0.0, 0.0), # first
(0.5, 49.0), # median, nearest-rank
(1.0, 99.0), # last
])
def test_other_fractions(self, fraction, expected):
assert _percentile([float(i) for i in range(100)], fraction) == expected
class TestEdges:
def test_empty_is_zero_not_an_error(self):
# The loop calls this before any frame has been timed.
assert _percentile([], 0.99) == 0.0
def test_a_single_sample_is_itself(self):
assert _percentile([4.2], 0.99) == 4.2
def test_it_never_indexes_past_the_end(self):
for n in range(1, 50):
_percentile([float(i) for i in range(n)], 1.0) # must not raise
class TestItSaysSomethingUsefulAboutFrames:
def test_one_freeze_does_not_drag_p99_up(self):
# 599 healthy frames and one 3.2s freeze: p99 should still describe
# the healthy population, while the worst frame is reported separately.
frames = [0.0083] * 599 + [3.2]
p99 = _percentile(sorted(frames), 0.99)
assert p99 == pytest.approx(0.0083), p99
assert max(frames) == 3.2
def test_sustained_slowness_does_move_it(self):
# Ten percent of frames slow is not an outlier, it is the shape of the
# distribution, and p99 must reflect that.
frames = [0.0083] * 540 + [0.05] * 60
assert _percentile(sorted(frames), 0.99) == pytest.approx(0.05)
-300
View File
@@ -1,300 +0,0 @@
"""Tests that live content can take extra turns inside the Vegas ticker.
Vegas was a strict round robin -- every plugin exactly once per cycle -- and
live content did not appear in it at all, because the display controller
refused to run the ticker while anything was live. With a dozen plugins
enabled that left a live score either absent or minutes stale.
Two things change, both off by default. `live_in_ticker` keeps the marquee
running instead of yielding to a full-screen takeover, and the rotation is
expanded by Smooth Weighted Round-Robin so a weighted plugin gets several
slots per cycle, spaced through it rather than clumped.
Weights are per plugin, not per game: a scoreboard showing four live games
still occupies one slot at a time and rotates its own games within it.
"""
from unittest.mock import Mock
import pytest
from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.stream_manager import StreamManager
class FakePlugin:
"""A plugin that can fail in each place independently.
hook_raises and live_raises are separate because they mean different
things: a broken weight calculation should still leave the core's own
live-content check usable, while a plugin that cannot answer whether it is
live at all has nothing left to fall back on.
"""
def __init__(self, live=False, declared=None, raises=False,
hook_raises=False, live_raises=False):
self._live = live
self._declared = declared
self._hook_raises = hook_raises or raises
self._live_raises = live_raises or raises
self.enabled = True
def has_live_priority(self):
if self._live_raises:
raise RuntimeError("cannot say whether I am live")
return self._live
def has_live_content(self):
return self._live
def get_vegas_priority_weight(self):
if self._hook_raises:
raise RuntimeError("weight calculation blew up")
return self._declared
def _manager(plugins, **cfg):
config = VegasModeConfig(live_in_ticker=cfg.pop('live_in_ticker', True), **cfg)
pm = Mock()
pm.plugins = plugins
sm = StreamManager.__new__(StreamManager)
sm.config = config
sm.plugin_manager = pm
return sm
def _counts(schedule):
return {p: schedule.count(p) for p in set(schedule)}
def _max_gap(schedule, plugin_id):
"""Largest gap between consecutive appearances, wrapping around."""
at = [i for i, p in enumerate(schedule) if p == plugin_id]
if len(at) < 2:
return len(schedule)
gaps = [b - a for a, b in zip(at, at[1:])]
gaps.append(len(schedule) - at[-1] + at[0])
return max(gaps)
class TestWeightsComeFromTheRightPlace:
def test_a_quiet_plugin_gets_one_slot(self):
sm = _manager({'clock': FakePlugin()})
assert sm._plugin_weight('clock') == 1
def test_live_content_earns_the_configured_weight(self):
sm = _manager({'mlb': FakePlugin(live=True)}, live_weight=4)
assert sm._plugin_weight('mlb') == 4
def test_a_plugin_may_answer_for_itself(self):
# The only route for favorite-team awareness: the core can see that a
# game is live, not whose.
sm = _manager({'mlb': FakePlugin(live=True, declared=7)}, live_weight=3)
assert sm._plugin_weight('mlb') == 7
def test_declaring_none_defers_to_the_core(self):
sm = _manager({'mlb': FakePlugin(live=True, declared=None)}, live_weight=3)
assert sm._plugin_weight('mlb') == 3
def test_a_declared_weight_is_clamped(self):
sm = _manager({'a': FakePlugin(declared=99), 'b': FakePlugin(declared=0)})
assert sm._plugin_weight('a') == 10
assert sm._plugin_weight('b') == 1
def test_a_plugin_that_raises_everywhere_weighs_one(self):
sm = _manager({'bad': FakePlugin(raises=True)})
assert sm._plugin_weight('bad') == 1
def test_a_broken_hook_still_earns_the_live_boost(self):
# The hook is only how a plugin asks for *more* than live_weight.
# Losing it should cost the favorite distinction, not the live boost:
# has_live_priority/has_live_content are separate and still work.
sm = _manager({'mlb': FakePlugin(live=True, hook_raises=True)},
live_weight=4)
assert sm._plugin_weight('mlb') == 4
def test_a_broken_hook_on_a_quiet_plugin_weighs_one(self):
sm = _manager({'clock': FakePlugin(live=False, hook_raises=True)},
live_weight=4)
assert sm._plugin_weight('clock') == 1
def test_a_plugin_that_cannot_say_whether_it_is_live_weighs_one(self):
# Nothing left to fall back on, so no boost.
sm = _manager({'mlb': FakePlugin(live=True, live_raises=True)},
live_weight=4)
assert sm._plugin_weight('mlb') == 1
def test_an_unknown_plugin_weighs_one(self):
assert _manager({})._plugin_weight('ghost') == 1
class TestTheSchedule:
def test_nothing_weighted_leaves_the_order_untouched(self):
order = ['weather', 'clock', 'news']
sm = _manager({p: FakePlugin() for p in order})
assert sm._apply_priority_weights(order) == order
def test_off_by_default_the_order_is_untouched(self):
order = ['weather', 'mlb', 'news']
sm = _manager({'weather': FakePlugin(), 'mlb': FakePlugin(live=True),
'news': FakePlugin()}, live_in_ticker=False, live_weight=3)
assert sm._apply_priority_weights(order) == order
def test_a_live_plugin_takes_its_share_of_slots(self):
order = ['weather', 'mlb', 'news', 'clock']
sm = _manager({'weather': FakePlugin(), 'mlb': FakePlugin(live=True),
'news': FakePlugin(), 'clock': FakePlugin()},
live_weight=3)
schedule = sm._apply_priority_weights(order)
counts = _counts(schedule)
assert counts['mlb'] == 3, counts
assert counts['weather'] == counts['news'] == counts['clock'] == 1, counts
assert len(schedule) == 6
def test_every_plugin_still_appears(self):
# A boost must not starve anything out of the cycle.
order = ['a', 'b', 'c', 'd', 'e', 'f']
plugins = {p: FakePlugin() for p in order}
plugins['a'] = FakePlugin(live=True, declared=10)
sm = _manager(plugins)
schedule = sm._apply_priority_weights(order)
assert set(schedule) == set(order), set(order) - set(schedule)
def test_nothing_doubles_across_the_cycle_seam(self):
# The strip loops, so the last slot neighbours the first. Smooth
# Weighted Round-Robin schedules the heaviest item first and often
# last too, which put the one clump the algorithm exists to avoid at
# the one place a within-cycle check cannot see.
order = ['baseball', 'weather', 'geochron', 'flights', 'stocks',
'oftheday', 'youtube', 'stocknews', 'leaderboard',
'countdown', 'odds', 'f1', 'football', 'music']
plugins = {p: FakePlugin() for p in order}
plugins['baseball'] = FakePlugin(live=True, declared=5)
plugins['football'] = FakePlugin(live=True, declared=3)
schedule = _manager(plugins)._apply_priority_weights(order)
n = len(schedule)
doubles = [schedule[i] for i in range(n)
if schedule[i] == schedule[(i + 1) % n]]
assert not doubles, "%r repeats across the seam in %r" % (doubles, schedule)
def test_the_seam_repair_keeps_every_slot(self):
order = ['a', 'b', 'c', 'd', 'e', 'f']
plugins = {p: FakePlugin() for p in order}
plugins['a'] = FakePlugin(live=True, declared=4)
schedule = _manager(plugins)._apply_priority_weights(order)
assert _counts(schedule)['a'] == 4, _counts(schedule)
assert sorted(schedule) == sorted(
['a'] * 4 + ['b', 'c', 'd', 'e', 'f']), schedule
def test_the_repair_uses_the_widest_gap(self):
# Moving the trailing repeat into the first slot that merely fits
# undoes the spacing: on a 28-slot rotation that turned a gap of 7
# into a gap of 2, which is more clumped than the seam ever was.
order = ['a'] + ['p%d' % i for i in range(13)]
plugins = {p: FakePlugin() for p in order}
plugins['a'] = FakePlugin(live=True, declared=4)
schedule = _manager(plugins)._apply_priority_weights(order)
at = [i for i, p in enumerate(schedule) if p == 'a']
gaps = [b - a for a, b in zip(at, at[1:])]
gaps.append(len(schedule) - at[-1] + at[0])
ideal = len(schedule) / len(at)
assert min(gaps) >= ideal / 2, "gaps %r for ideal %.1f" % (gaps, ideal)
def test_an_unavoidable_double_is_left_alone(self):
# Five of seven slots are the same plugin, so it must neighbour
# itself. Better to schedule it than to refuse or loop forever.
order = ['a', 'b', 'c']
plugins = {p: FakePlugin() for p in order}
plugins['a'] = FakePlugin(live=True, declared=5)
schedule = _manager(plugins)._apply_priority_weights(order)
assert _counts(schedule) == {'a': 5, 'b': 1, 'c': 1}, _counts(schedule)
assert set(schedule) == {'a', 'b', 'c'}
def test_the_repair_never_creates_a_new_double(self):
# The first version guarded the slot the repeated value moves *into*
# but not the one the displaced element lands in, so this traded the
# seam duplicate for a fresh one and came back ending ['x', 'x'].
sm = _manager({})
out = sm._unclump_seam(['a', 'b', 'c', 'd', 'x', 'y', 'x', 'a'])
n = len(out)
doubles = [out[i] for i in range(n) if out[i] == out[(i + 1) % n]]
assert not doubles, "%r in %r" % (doubles, out)
assert sorted(out) == sorted(['a', 'b', 'c', 'd', 'x', 'y', 'x', 'a'])
def test_the_last_two_slots_are_a_usable_swap(self):
# Reasoning about indices said this candidate was unsafe because
# schedule[j] is schedule[-2]; after the swap its neighbour is the
# repeated value, not itself. Refusing it left the only repair this
# schedule has on the table.
assert _manager({})._unclump_seam(['a', 'b', 'c', 'a']) == ['a', 'b', 'a', 'c']
def test_no_seam_schedule_is_ever_made_worse(self):
import random
sm = _manager({})
random.seed(11)
checked = 0
for size in range(3, 10):
for _ in range(400):
original = [random.choice('abcd') for _ in range(size)]
if original[0] != original[-1]:
continue
checked += 1
out = sm._unclump_seam(list(original))
n = len(out)
before = sum(1 for i in range(n)
if original[i] == original[(i + 1) % n])
after = sum(1 for i in range(n) if out[i] == out[(i + 1) % n])
assert after <= before, (original, out)
assert sorted(out) == sorted(original), (original, out)
assert checked > 100, "the generator stopped producing seam cases"
def test_a_schedule_too_short_to_repair_is_returned_as_is(self):
sm = _manager({})
assert sm._unclump_seam(['a', 'a']) == ['a', 'a']
assert sm._unclump_seam(['a']) == ['a']
assert sm._unclump_seam([]) == []
def test_a_schedule_with_no_seam_clash_is_untouched(self):
sm = _manager({})
plain = ['a', 'b', 'c', 'a', 'd']
assert sm._unclump_seam(plain) == plain
def test_repeats_are_spread_not_clumped(self):
# The point of Smooth Weighted Round-Robin. Three-in-a-row followed by
# a long silence would be worse than not boosting at all.
order = ['weather', 'mlb', 'news', 'clock', 'stocks', 'f1']
plugins = {p: FakePlugin() for p in order}
plugins['mlb'] = FakePlugin(live=True)
sm = _manager(plugins, live_weight=3)
schedule = sm._apply_priority_weights(order)
assert _counts(schedule)['mlb'] == 3
# Evenly spread over 8 slots means a gap of about 3, never 6.
assert _max_gap(schedule, 'mlb') <= 4, schedule
# And never twice running.
assert not any(a == b == 'mlb' for a, b in zip(schedule, schedule[1:])), schedule
def test_a_favorite_outranks_another_live_game(self):
order = ['weather', 'mlb', 'nhl']
sm = _manager({'weather': FakePlugin(),
'mlb': FakePlugin(live=True, declared=5),
'nhl': FakePlugin(live=True)}, live_weight=2)
counts = _counts(sm._apply_priority_weights(order))
assert counts['mlb'] == 5 and counts['nhl'] == 2 and counts['weather'] == 1, counts
def test_an_empty_rotation_is_harmless(self):
assert _manager({})._apply_priority_weights([]) == []
class TestConfigParsing:
def test_defaults_preserve_todays_behaviour(self):
cfg = VegasModeConfig.from_config({})
assert cfg.live_in_ticker is False
assert cfg.live_weight == 3 and cfg.favorite_live_weight == 5
@pytest.mark.parametrize("given,expected", [(0, 1), (-4, 1), (99, 10), (4, 4)])
def test_weights_are_clamped(self, given, expected):
cfg = VegasModeConfig.from_config(
{'display': {'vegas_scroll': {'live_weight': given}}})
assert cfg.live_weight == expected
-248
View File
@@ -1,248 +0,0 @@
"""Tests for surfacing the underlying error in web responses.
Regression under test: every failing endpoint returned "An error occurred; see
logs for details" and nothing else. On a device whose storage was failing that
sentence came back from the restart action, from /system/status, and from
/logs -- the log viewer itself -- because journalctl could not be executed. The
exception underneath said `[Errno 5] Input/output error: 'systemctl'`, which
names the fault outright, and nine handlers were discarding it entirely rather
than even logging it.
"""
import pytest
from src.web_interface.error_handler import describe_exception
class TestDescribeException:
def test_names_the_type_and_message(self):
detail = describe_exception(OSError(5, "Input/output error", "systemctl"))
assert detail == "OSError: [Errno 5] Input/output error: 'systemctl'"
def test_the_reported_failure_is_legible(self):
# The whole point: this string is the diagnosis.
assert "Input/output error" in describe_exception(
OSError(5, "Input/output error", "systemctl"))
def test_a_bare_exception_still_names_its_type(self):
# A PermissionError with no message still says more than "unknown".
assert describe_exception(PermissionError()) == "PermissionError"
assert describe_exception(Exception()) == "Exception"
def test_message_is_kept_when_present(self):
assert describe_exception(ValueError("bad port")) == "ValueError: bad port"
class TestCredentialRedaction:
"""Exception text quotes URLs, and plugins authenticate by query string."""
@pytest.mark.parametrize("secret_text,leaked", [
("failed: https://api.x.com/v1?api_key=SEC123&city=Tampa", "SEC123"),
("token=abcdef123456 was rejected", "abcdef123456"),
("connect failed password=hunter2", "hunter2"),
("GET /?access_token=zzz999", "zzz999"),
('{"secret": "topsecret"}', "topsecret"),
# requests quotes the URL it failed on, and both of these forms turn
# up in real client exceptions.
("401 for https://user:hunter2@example.com/api", "hunter2"),
("headers: {'Authorization': 'Bearer eyJ.SECRET.sig'}", "eyJ.SECRET.sig"),
("Authorization: Basic dXNlcjpwYXNzd29yZA==", "dXNlcjpwYXNzd29yZA=="),
("Proxy-Authorization: Bearer ptok999", "ptok999"),
# Any scheme, not a fixed list -- a list silently leaks whatever it
# does not name, and plugin APIs invent their own.
("Authorization: ApiKey SECRET123", "SECRET123"),
("Authorization: Negotiate YIIZnegotiateblob", "YIIZnegotiateblob"),
("Authorization: NTLM TlRMTVNTUAAB", "TlRMTVNTUAAB"),
("authorization: barecredential", "barecredential"),
])
def test_credentials_never_reach_the_response(self, secret_text, leaked):
detail = describe_exception(RuntimeError(secret_text))
assert leaked not in detail
assert "<redacted>" in detail
def test_the_parameter_name_survives_redaction(self):
# Knowing *which* credential was involved is part of the diagnosis.
detail = describe_exception(RuntimeError("https://x/y?api_key=SEC123"))
assert "api_key" in detail
def test_unknown_schemes_keep_their_name(self):
for scheme in ("ApiKey", "Negotiate", "NTLM", "AWS4-HMAC-SHA256"):
detail = describe_exception(
RuntimeError("Authorization: %s SECRETVALUE" % scheme))
assert scheme in detail, detail
assert "SECRETVALUE" not in detail, detail
def test_auth_scheme_and_username_survive(self):
# Which kind of credential, and whose, without the credential itself.
assert "Bearer" in describe_exception(
RuntimeError("Authorization: Bearer eyJ.SECRET.sig"))
assert "user" in describe_exception(
RuntimeError("https://user:hunter2@example.com"))
def test_non_secret_context_is_preserved(self):
detail = describe_exception(RuntimeError("https://api.x.com/v1?city=Tampa"))
assert "city=Tampa" in detail
assert "<redacted>" not in detail
class TestBounds:
def test_long_messages_are_truncated(self):
detail = describe_exception(ValueError("x" * 5000))
assert len(detail) <= 400
def test_newlines_are_collapsed_to_one_line(self):
detail = describe_exception(ValueError("line one\nline two\tthree"))
assert "\n" not in detail and "\t" not in detail
assert detail == "ValueError: line one line two three"
def test_custom_length_is_honoured(self):
assert len(describe_exception(ValueError("y" * 500), max_length=50)) <= 50
class TestHandlersCarryDetail:
"""The response shape callers actually see."""
def test_no_api_v3_handler_discards_its_exception(self):
"""Every generic-message handler must log a traceback and return detail.
Nine of them bound `e` and never used it, so the promised log entry was
never written either. Checking merely that *something* was logged is
too weak -- a `logger.info("failed")` would satisfy it while throwing
the exception away just as completely, so this asserts the two things
that actually make the failure diagnosable: an error-level record with
the traceback, and the sanitized detail in the response.
"""
import ast
src = open("web_interface/blueprints/api_v3.py").read()
tree = ast.parse(src)
generic = "An error occurred; see logs for details"
def logs_a_traceback(handler):
"""An error/exception-level log call carrying exc_info."""
for call in [n for n in ast.walk(handler) if isinstance(n, ast.Call)]:
func = call.func
if not isinstance(func, ast.Attribute):
continue
if func.attr == "exception": # implies exc_info
return True
if func.attr not in ("error", "critical"):
continue
if any(kw.arg == "exc_info" and getattr(kw.value, "value", False) is True
for kw in call.keywords):
return True
return False
def describes_this_exception(node, bound):
"""A describe_exception(<bound>) call anywhere under `node`."""
for call in [n for n in ast.walk(node) if isinstance(n, ast.Call)]:
if not (isinstance(call.func, ast.Name)
and call.func.id == "describe_exception"):
continue
if bound is None:
return True # bare `except:` cannot name it; accept
if any(isinstance(a, ast.Name) and a.id == bound
for a in call.args):
return True
return False
def returns_the_detail(handler):
"""The detail must be inside what the handler actually returns.
Looking anywhere in the handler is too weak: a handler could
compute describe_exception(e), drop it on the floor, and return the
generic message with no details field, while still passing. So the
call has to appear within a `return` expression.
"""
returns = [n for n in ast.walk(handler) if isinstance(n, ast.Return)]
if not returns:
return False
return all(describes_this_exception(r, handler.name) for r in returns)
offenders = []
for h in [n for n in ast.walk(tree) if isinstance(n, ast.ExceptHandler)]:
seg = ast.get_source_segment(src, h) or ""
if generic not in seg:
continue
missing = []
if not logs_a_traceback(h):
missing.append("error-level log with exc_info")
if not returns_the_detail(h):
missing.append("describe_exception(e) in the response")
if missing:
offenders.append((h.lineno, missing))
assert not offenders, (
"handlers returning the generic message without %s: %r"
% ("both a traceback log and the detail", offenders))
def test_client_errors_keep_their_own_status(self):
"""A 405 must not be reported as a server-side UNKNOWN_ERROR.
Werkzeug's HTTPExceptions subclass Exception, so the catch-all saw them
too: a GET on a POST-only route came back 500 "an error occurred",
which tells the caller nothing and blames the wrong side. Found while
probing a device whose POST-only config endpoints answered every GET
with UNKNOWN_ERROR.
"""
from flask import Flask, jsonify
from werkzeug.exceptions import HTTPException
app = Flask(__name__)
@app.errorhandler(Exception)
def handle(error):
if isinstance(error, HTTPException):
return jsonify({
"status": "error",
"error_code": (error.name or "HTTP_ERROR").upper().replace(" ", "_"),
"message": error.description,
}), error.code or 500
return jsonify({
"status": "error",
"error_code": "UNKNOWN_ERROR",
"message": "An error occurred; see logs for details",
"details": describe_exception(error),
}), 500
@app.route("/only-post", methods=["POST"])
def only_post():
return jsonify({"ok": True})
@app.route("/boom")
def boom():
raise OSError(5, "Input/output error", "systemctl")
client = app.test_client()
resp = client.get("/only-post")
assert resp.status_code == 405, "a wrong method must stay a 405"
assert resp.get_json()["error_code"] == "METHOD_NOT_ALLOWED"
# A genuine server fault still reports as one, with its detail.
resp = client.get("/boom")
assert resp.status_code == 500
assert "Input/output error" in resp.get_json()["details"]
def test_global_handler_reports_the_underlying_error(self):
from flask import Flask, jsonify
app = Flask(__name__)
@app.errorhandler(Exception)
def handle(error):
return jsonify({
"status": "error",
"error_code": "UNKNOWN_ERROR",
"message": "An error occurred; see logs for details",
"details": describe_exception(error),
}), 500
@app.route("/boom")
def boom():
raise OSError(5, "Input/output error", "systemctl")
client = app.test_client()
body = client.get("/boom").get_json()
assert body["error_code"] == "UNKNOWN_ERROR"
assert "Input/output error" in body["details"]
@@ -1,408 +0,0 @@
"""Tests the calendar plugin's OAuth and calendar-listing endpoints.
The plugin's config UI advertised a three-step setup, but only step 1 existed
on the server. Step 3's picker fetched /api/v3/plugins/calendar/list-calendars,
which was never registered, so Flask fell through to the global 404 handler and
the user saw "Resource not found" with nothing to say which resource. Step 2
had no endpoint either, and no field in the schema at all, even though the
plugin ships calendar_registration.py written expressly for a web-driven
two-step flow.
These cover the two new routes: that they exist, that they fail with something
actionable rather than a bare 404, and that the shapes the widgets consume are
what the server actually sends.
"""
import json
import pickle
import sys
from pathlib import Path
import pytest
project_root = Path(__file__).parent.parent.parent
sys.path.insert(0, str(project_root))
from web_interface.blueprints import api_v3 as mod # noqa: E402
@pytest.fixture
def client(monkeypatch, tmp_path):
"""A test client whose calendar plugin lives in tmp_path."""
from flask import Flask
plugin_dir = tmp_path / 'calendar'
plugin_dir.mkdir()
app = Flask(__name__)
app.register_blueprint(mod.api_v3, url_prefix='/api/v3')
app.config['TESTING'] = True
monkeypatch.setattr(mod, '_calendar_plugin_dir', lambda: plugin_dir)
with app.test_client() as c:
c.plugin_dir = plugin_dir
yield c
@pytest.fixture
def uninstalled(monkeypatch):
from flask import Flask
app = Flask(__name__)
app.register_blueprint(mod.api_v3, url_prefix='/api/v3')
app.config['TESTING'] = True
monkeypatch.setattr(mod, '_calendar_plugin_dir', lambda: None)
with app.test_client() as c:
yield c
class TestTheRoutesExistAtAll:
"""The original bug: the URLs the widgets call were not registered."""
def test_list_calendars_is_routed(self, client):
response = client.get('/api/v3/plugins/calendar/list-calendars')
# Reaching the handler is the whole point; what it then says about
# missing setup is TestItSaysWhatIsWrong's business.
assert response.status_code != 404, "still unrouted"
assert response.get_json()['message'] != 'Resource not found'
def test_authenticate_is_routed(self, client):
response = client.post('/api/v3/plugins/calendar/authenticate', json={})
assert response.status_code != 404, "still unrouted"
assert response.get_json()['message'] != 'Resource not found'
def test_both_urls_match_what_the_widgets_request(self):
# The widgets hardcode these; a rename on either side reintroduces the
# original bug silently.
picker = Path(project_root) / 'web_interface/static/v3/js/widgets/google-calendar-picker.js'
oauth = Path(project_root) / 'web_interface/static/v3/js/widgets/google-oauth.js'
assert '/api/v3/plugins/calendar/list-calendars' in picker.read_text(encoding='utf-8')
assert '/api/v3/plugins/calendar/authenticate' in oauth.read_text(encoding='utf-8')
source = (Path(project_root) / 'web_interface/blueprints/api_v3.py').read_text(encoding='utf-8')
assert "'/plugins/calendar/list-calendars'" in source
assert "'/plugins/calendar/authenticate'" in source
def test_the_oauth_widget_is_dispatched_not_rendered_as_a_text_box(self):
# The string branch of the config template dispatches on an allow-list
# of widget names; anything missing from it silently falls through to a
# plain <input type="text">. That produced two boxes on the calendar
# page -- the widget's own, and a stray one for the same field -- and
# no way to tell which to paste into.
template = (Path(project_root)
/ 'web_interface/templates/v3/partials/plugin_config.html'
).read_text(encoding='utf-8')
allow_list_line = [ln for ln in template.splitlines()
if "str_widget in [" in ln]
assert allow_list_line, "the string widget allow-list moved"
assert "'google-oauth'" in allow_list_line[0], allow_list_line[0]
def test_the_widget_script_is_served(self):
base = (Path(project_root) / 'web_interface/templates/v3/base.html'
).read_text(encoding='utf-8')
assert 'widgets/google-oauth.js' in base
def test_the_status_line_is_announced(self):
# Every message the widget gives arrives after an async call, so a
# screen reader hears nothing unless the element is a live region.
widget = (Path(project_root)
/ 'web_interface/static/v3/js/widgets/google-oauth.js'
).read_text(encoding='utf-8')
# Both attributes must be on the *status* element. Searching for them
# separately would pass with each on a different node, which announces
# nothing.
assert "status.setAttribute('role', 'status')" in widget, widget[:0]
assert "status.setAttribute('aria-live', 'polite')" in widget
def test_the_paste_box_has_an_accessible_name(self):
# A visible label is not enough on its own: without the association the
# input's only name is a placeholder, which vanishes on focus -- which
# is exactly when the value is being pasted.
widget = (Path(project_root)
/ 'web_interface/static/v3/js/widgets/google-oauth.js'
).read_text(encoding='utf-8')
# The binding is what matters, not that both lines exist: a `for` and
# an `id` that disagree leave the input just as anonymous. Both must
# go through the same identifier.
import re as _re
for_target = _re.search(r"codeLabel\.setAttribute\('for',\s*(\w+)\)", widget)
id_source = _re.search(r"codeInput\.id\s*=\s*(\w+)", widget)
assert for_target and id_source, (for_target, id_source)
assert for_target.group(1) == id_source.group(1), (
"label points at %r but the input is %r"
% (for_target.group(1), id_source.group(1)))
def test_the_failed_page_is_called_out_loudly(self):
# The loopback redirect lands on a browser error page at exactly the
# moment the user has to act. In small grey text it gets missed and the
# flow reads as broken while it is working.
widget = (Path(project_root)
/ 'web_interface/static/v3/js/widgets/google-oauth.js'
).read_text(encoding='utf-8')
assert 'expected' in widget.lower()
assert 'amber' in widget, "the warning is not visually distinguished"
class TestItSaysWhatIsWrong:
def test_listing_without_a_token_asks_for_step_2(self, client):
response = client.get('/api/v3/plugins/calendar/list-calendars')
assert response.status_code == 400
body = response.get_json()
assert body['status'] == 'error'
assert 'step 2' in body['message'].lower(), body['message']
def test_authenticating_without_credentials_asks_for_step_1(self, client):
response = client.post('/api/v3/plugins/calendar/authenticate', json={})
assert response.status_code == 400
assert 'step 1' in response.get_json()['message'].lower()
def test_an_uninstalled_plugin_says_so(self, uninstalled):
for response in (
uninstalled.get('/api/v3/plugins/calendar/list-calendars'),
uninstalled.post('/api/v3/plugins/calendar/authenticate', json={}),
):
assert response.status_code == 404
# A 404 here is honest -- but it must name the plugin, not read as
# the generic "Resource not found" that started this.
assert 'not installed' in response.get_json()['message'].lower()
class TestTheScriptRunner:
def test_it_returns_the_json_the_script_prints(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text(
'print(\'{"status": "success", "auth_url": "https://x"}\')\n',
encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert error is None
assert payload['auth_url'] == 'https://x'
def test_it_ignores_noise_before_the_json(self, tmp_path):
# An import warning or a library writing to stdout would otherwise
# make the last-line parse fail.
script = tmp_path / 'calendar_registration.py'
script.write_text(
'print("some library warning")\n'
'print(\'{"status": "success"}\')\n', encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert error is None and payload['status'] == 'success'
def test_it_passes_stdin_through(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text(
'import sys, json\n'
'print(json.dumps({"status": "success", "got": sys.stdin.read().strip()}))\n',
encoding='utf-8')
payload, _ = mod._run_calendar_registration(tmp_path, 'http://127.0.0.1/?code=abc')
assert payload['got'] == 'http://127.0.0.1/?code=abc'
def test_a_missing_script_is_reported(self, tmp_path):
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'script not found' in error.lower()
def test_output_that_is_not_json_is_reported_with_context(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text('import sys\nsys.stderr.write("boom\\n")\n', encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'no result' in error.lower()
assert 'boom' in error
class TestListingShape:
"""The picker reads cal.id, cal.summary and cal.primary."""
def _authenticate(self, client, monkeypatch, items):
creds = type('C', (), {'expired': False, 'refresh_token': None, 'valid': True})()
(client.plugin_dir / 'token.pickle').write_bytes(pickle.dumps({'x': 1}))
monkeypatch.setattr(mod.pickle if hasattr(mod, 'pickle') else pickle,
'loads', lambda *a, **k: creds, raising=False)
import types
fake_pickle = types.SimpleNamespace(load=lambda f: creds, dump=lambda *a: None)
# Callers pass a flat list of calendars; the API returns them wrapped
# in a page. One page is all these cases need -- TestPagination builds
# its own multi-page sequences.
pages = [{'items': items}]
state = {'i': 0}
def fake_list(**kwargs):
page = pages[min(state['i'], len(pages) - 1)]
state['i'] += 1
return types.SimpleNamespace(execute=lambda: page)
def fake_build(*args, **kwargs):
return types.SimpleNamespace(
calendarList=lambda: types.SimpleNamespace(list=fake_list))
real_import = __builtins__['__import__'] if isinstance(__builtins__, dict) \
else __builtins__.__import__
def fake_import(name, *args, **kwargs):
if name == 'pickle':
return fake_pickle
if name == 'google.auth.transport.requests':
return types.SimpleNamespace(Request=object)
if name == 'googleapiclient.discovery':
return types.SimpleNamespace(build=fake_build)
return real_import(name, *args, **kwargs)
monkeypatch.setattr('builtins.__import__', fake_import)
def test_it_returns_id_summary_and_primary(self, client, monkeypatch):
self._authenticate(client, monkeypatch, [
{'id': 'b@x', 'summary': 'Work'},
{'id': 'a@x', 'summary': 'Personal', 'primary': True},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['status'] == 'success'
assert {c['id'] for c in body['calendars']} == {'a@x', 'b@x'}
assert all(set(c) == {'id', 'summary', 'primary'} for c in body['calendars'])
def test_the_primary_calendar_comes_first(self, client, monkeypatch):
# Short list, but the one the user wants is almost always their own.
self._authenticate(client, monkeypatch, [
{'id': 'z@x', 'summary': 'Aardvarks'},
{'id': 'a@x', 'summary': 'Zebras', 'primary': True},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['calendars'][0]['id'] == 'a@x'
assert body['calendars'][0]['primary'] is True
def test_a_calendar_without_a_name_still_lists(self, client, monkeypatch):
self._authenticate(client, monkeypatch, [{'id': 'noname@x'}])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['calendars'][0]['summary'] == 'noname@x'
def test_entries_without_an_id_are_dropped(self, client, monkeypatch):
# Nothing could be selected by such a row, and the checkbox value
# would be undefined.
self._authenticate(client, monkeypatch, [{'summary': 'ghost'}, {'id': 'real@x'}])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert [c['id'] for c in body['calendars']] == ['real@x']
class TestPagination:
"""calendarList.list pages at 250 and defaults to 100."""
def _paged(self, client, monkeypatch, pages):
import types
creds = type('C', (), {'expired': False, 'refresh_token': None, 'valid': True})()
(client.plugin_dir / 'token.pickle').write_bytes(b'x')
state = {'i': 0}
seen = []
def fake_list(**kwargs):
seen.append(kwargs)
page = pages[min(state['i'], len(pages) - 1)]
state['i'] += 1
return types.SimpleNamespace(execute=lambda: page)
def fake_build(*args, **kwargs):
return types.SimpleNamespace(
calendarList=lambda: types.SimpleNamespace(list=fake_list))
real_import = __builtins__['__import__'] if isinstance(__builtins__, dict) \
else __builtins__.__import__
def fake_import(name, *args, **kwargs):
if name == 'pickle':
return types.SimpleNamespace(load=lambda f: creds, dump=lambda *a: None)
if name == 'google.auth.transport.requests':
return types.SimpleNamespace(Request=object)
if name == 'googleapiclient.discovery':
return types.SimpleNamespace(build=fake_build)
return real_import(name, *args, **kwargs)
monkeypatch.setattr('builtins.__import__', fake_import)
return seen
def test_every_page_is_collected(self, client, monkeypatch):
# Taking only the first page would hide calendars from the picker with
# nothing to say the list was cut short.
self._paged(client, monkeypatch, [
{'items': [{'id': 'a@x', 'summary': 'A'}], 'nextPageToken': 't1'},
{'items': [{'id': 'b@x', 'summary': 'B'}], 'nextPageToken': 't2'},
{'items': [{'id': 'c@x', 'summary': 'C'}]},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert [c['id'] for c in body['calendars']] == ['a@x', 'b@x', 'c@x']
def test_the_page_token_is_passed_back(self, client, monkeypatch):
seen = self._paged(client, monkeypatch, [
{'items': [{'id': 'a@x', 'summary': 'A'}], 'nextPageToken': 'tok'},
{'items': [{'id': 'b@x', 'summary': 'B'}]},
])
client.get('/api/v3/plugins/calendar/list-calendars')
assert seen[0]['pageToken'] is None
assert seen[1]['pageToken'] == 'tok'
assert all(k['maxResults'] == 250 for k in seen)
def test_a_looping_token_cannot_spin_forever(self, client, monkeypatch):
# Every page claims another follows.
self._paged(client, monkeypatch, [
{'items': [{'id': 'a@x', 'summary': 'A'}], 'nextPageToken': 'same'},
])
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert body['status'] == 'success'
assert len(body['calendars']) <= mod._CALENDAR_LIST_MAX_PAGES
class TestDiagnosticsAreRedacted:
def test_script_stderr_is_redacted_on_the_way_out(self, tmp_path):
script = tmp_path / 'calendar_registration.py'
script.write_text(
'import sys\n'
'sys.stderr.write("boom client_secret=hunter2 more\\n")\n',
encoding='utf-8')
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'hunter2' not in error, error
assert '<redacted>' in error, error
def test_a_failing_script_payload_is_redacted(self, client):
(client.plugin_dir / 'credentials.json').write_text('{}', encoding='utf-8')
(client.plugin_dir / 'calendar_registration.py').write_text(
'import json\n'
'print(json.dumps({"status": "error", '
'"message": "Failed: client_secret=topsecret"}))\n',
encoding='utf-8')
body = client.post('/api/v3/plugins/calendar/authenticate',
json={}).get_json()
assert body['status'] == 'error'
assert 'topsecret' not in json.dumps(body), body
assert '<redacted>' in body['message'], body
def test_an_unrunnable_script_is_reported_without_raw_exception_text(self,
tmp_path,
monkeypatch):
# OSError from the spawn carries the interpreter path and whatever the
# OS chose to say; it reaches the client through the redactor like
# everything else.
script = tmp_path / 'calendar_registration.py'
script.write_text('', encoding='utf-8')
def boom(*a, **k):
raise OSError("Exec format error: token=abcd1234 /usr/bin/python3")
monkeypatch.setattr(mod.subprocess, 'run', boom)
payload, error = mod._run_calendar_registration(tmp_path, '')
assert payload is None
assert 'abcd1234' not in error, error
assert 'OSError' in error, error
def test_a_missing_google_library_is_reported_without_raw_exception_text(
self, client, monkeypatch):
(client.plugin_dir / 'token.pickle').write_bytes(b'x')
real_import = __builtins__['__import__'] if isinstance(__builtins__, dict) \
else __builtins__.__import__
def fake_import(name, *args, **kwargs):
if name.startswith('google'):
raise ImportError("No module named 'google' password=hunter2")
return real_import(name, *args, **kwargs)
monkeypatch.setattr('builtins.__import__', fake_import)
body = client.get('/api/v3/plugins/calendar/list-calendars').get_json()
assert 'hunter2' not in json.dumps(body), body
assert 'requirements.txt' in body['message']
+3 -33
View File
@@ -16,8 +16,6 @@ from datetime import datetime, timedelta
sys.path.insert(0, str(Path(__file__).parent.parent)) sys.path.insert(0, str(Path(__file__).parent.parent))
from src.config_manager import ConfigManager from src.config_manager import ConfigManager
from src.web_interface.error_handler import describe_exception
from werkzeug.exceptions import HTTPException
from src.exceptions import ConfigError from src.exceptions import ConfigError
from src.plugin_system.plugin_manager import PluginManager from src.plugin_system.plugin_manager import PluginManager
from src.plugin_system.store_manager import PluginStoreManager from src.plugin_system.store_manager import PluginStoreManager
@@ -393,42 +391,15 @@ def internal_error(error):
import logging import logging
logger = logging.getLogger('web_interface') logger = logging.getLogger('web_interface')
logger.error("Internal server error", exc_info=True) logger.error("Internal server error", exc_info=True)
payload = { return jsonify({
'status': 'error', 'status': 'error',
'error_code': 'INTERNAL_ERROR', 'error_code': 'INTERNAL_ERROR',
'message': 'An internal error occurred; see logs for details', 'message': 'An internal error occurred; see logs for details',
} }), 500
# Flask hands the original exception over as `error.original_exception`
# when propagation is off; without it there is nothing to describe.
original = getattr(error, 'original_exception', None) or (
error if isinstance(error, BaseException) else None)
if original is not None:
payload['details'] = describe_exception(original)
return jsonify(payload), 500
@app.errorhandler(Exception) @app.errorhandler(Exception)
def handle_exception(error): def handle_exception(error):
"""Handle all unhandled exceptions. """Handle all unhandled exceptions."""
Returning only "see logs for details" is fine until the logs are exactly
what you cannot reach. A device with failing storage answered every
endpoint with that sentence -- including the log viewer, because journalctl
could not be executed -- while the exception underneath said
`[Errno 5] Input/output error`. Naming the error costs nothing here and is
frequently the whole diagnosis, so include it alongside the log pointer.
"""
# Werkzeug's HTTPExceptions subclass Exception, so this catch-all sees
# them too and was reporting every 405, 400, 413 and 415 as a server-side
# UNKNOWN_ERROR 500. A GET on a POST-only route came back as "an error
# occurred" rather than "method not allowed", which tells the caller
# nothing and blames the wrong side. Hand those back as themselves.
if isinstance(error, HTTPException):
return jsonify({
'status': 'error',
'error_code': (error.name or 'HTTP_ERROR').upper().replace(' ', '_'),
'message': error.description,
}), error.code or 500
import logging import logging
logger = logging.getLogger('web_interface') logger = logging.getLogger('web_interface')
logger.error("Unhandled exception", exc_info=True) logger.error("Unhandled exception", exc_info=True)
@@ -436,7 +407,6 @@ def handle_exception(error):
'status': 'error', 'status': 'error',
'error_code': 'UNKNOWN_ERROR', 'error_code': 'UNKNOWN_ERROR',
'message': 'An error occurred; see logs for details', 'message': 'An error occurred; see logs for details',
'details': describe_exception(error),
}), 500 }), 500
# Captive portal redirect middleware # Captive portal redirect middleware
+70 -319
View File
@@ -22,7 +22,6 @@ logger = logging.getLogger(__name__)
from src.web_interface.api_helpers import success_response, error_response, validate_request_json from src.web_interface.api_helpers import success_response, error_response, validate_request_json
from src.web_interface.errors import ErrorCode from src.web_interface.errors import ErrorCode
from src.web_interface.secret_helpers import find_secret_fields, separate_secrets from src.web_interface.secret_helpers import find_secret_fields, separate_secrets
from src.web_interface.error_handler import describe_exception, redact_text
from src.plugin_system.operation_types import OperationType from src.plugin_system.operation_types import OperationType
from src.web_interface.validators import ( from src.web_interface.validators import (
validate_file_upload validate_file_upload
@@ -273,7 +272,7 @@ def get_main_config():
return jsonify({'status': 'success', 'data': config}) return jsonify({'status': 'success', 'data': config})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/config/schedule', methods=['GET']) @api_v3.route('/config/schedule', methods=['GET'])
def get_schedule_config(): def get_schedule_config():
@@ -291,11 +290,9 @@ def get_schedule_config():
return success_response(data=schedule_config) return success_response(data=schedule_config)
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return error_response( return error_response(
ErrorCode.CONFIG_LOAD_FAILED, ErrorCode.CONFIG_LOAD_FAILED,
"An error occurred; see logs for details", "An error occurred; see logs for details",
details=describe_exception(e),
status_code=500 status_code=500
) )
@@ -471,7 +468,7 @@ def save_schedule_config():
ErrorCode.CONFIG_SAVE_FAILED, ErrorCode.CONFIG_SAVE_FAILED,
"An error occurred; see logs for details", "An error occurred; see logs for details",
status_code=500, details=describe_exception(e) status_code=500
) )
@api_v3.route('/config/dim-schedule', methods=['GET']) @api_v3.route('/config/dim-schedule', methods=['GET'])
@@ -519,14 +516,14 @@ def get_dim_schedule_config():
return error_response( return error_response(
ErrorCode.CONFIG_LOAD_FAILED, ErrorCode.CONFIG_LOAD_FAILED,
"An error occurred; see logs for details", "An error occurred; see logs for details",
status_code=500, details=describe_exception(e) status_code=500
) )
except Exception as e: except Exception as e:
logging.error(f"[DIM SCHEDULE] Unexpected error loading config: {e}", exc_info=True) logging.error(f"[DIM SCHEDULE] Unexpected error loading config: {e}", exc_info=True)
return error_response( return error_response(
ErrorCode.CONFIG_LOAD_FAILED, ErrorCode.CONFIG_LOAD_FAILED,
"An error occurred; see logs for details", "An error occurred; see logs for details",
status_code=500, details=describe_exception(e) status_code=500
) )
@api_v3.route('/config/dim-schedule', methods=['POST']) @api_v3.route('/config/dim-schedule', methods=['POST'])
@@ -690,7 +687,7 @@ def save_dim_schedule_config():
ErrorCode.CONFIG_SAVE_FAILED, ErrorCode.CONFIG_SAVE_FAILED,
"An error occurred; see logs for details", "An error occurred; see logs for details",
status_code=500, details=describe_exception(e) status_code=500
) )
@api_v3.route('/config/main', methods=['POST']) @api_v3.route('/config/main', methods=['POST'])
@@ -796,7 +793,7 @@ def save_main_config():
'gpio_slowdown', 'rp1_rio', 'scan_mode', 'disable_hardware_pulsing', 'inverse_colors', 'show_refresh_rate', 'gpio_slowdown', 'rp1_rio', 'scan_mode', 'disable_hardware_pulsing', 'inverse_colors', 'show_refresh_rate',
'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz', 'use_short_date_format', 'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz', 'use_short_date_format',
'max_dynamic_duration_seconds', 'led_rgb_sequence', 'multiplexing', 'panel_type', 'max_dynamic_duration_seconds', 'led_rgb_sequence', 'multiplexing', 'panel_type',
'row_address_type', 'pixel_mapper_config', 'orientation'] 'row_address_type', 'pixel_mapper_config']
if any(k in data for k in display_fields): if any(k in data for k in display_fields):
if 'display' not in current_config: if 'display' not in current_config:
@@ -831,11 +828,6 @@ def save_main_config():
if 'pixel_mapper_config' in data and not isinstance(data['pixel_mapper_config'], str): if 'pixel_mapper_config' in data and not isinstance(data['pixel_mapper_config'], str):
return jsonify({'status': 'error', 'message': 'pixel_mapper_config must be a string (e.g. "U-mapper;Rotate:90" or empty)'}), 400 return jsonify({'status': 'error', 'message': 'pixel_mapper_config must be a string (e.g. "U-mapper;Rotate:90" or empty)'}), 400
# Validate orientation (physical mounting rotation; composed onto pixel_mapper_config at runtime)
ORIENTATION_ALLOWED = {'normal', '180'}
if 'orientation' in data and data['orientation'] not in ORIENTATION_ALLOWED:
return jsonify({'status': 'error', 'message': f"Invalid orientation '{data['orientation']}'. Allowed values: {', '.join(sorted(ORIENTATION_ALLOWED))}"}), 400
# Validate row_address_type # Validate row_address_type
if 'row_address_type' in data: if 'row_address_type' in data:
try: try:
@@ -849,7 +841,7 @@ def save_main_config():
for field in ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'hardware_mapping', 'scan_mode', for field in ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'hardware_mapping', 'scan_mode',
'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz', 'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz',
'led_rgb_sequence', 'multiplexing', 'panel_type', 'row_address_type', 'led_rgb_sequence', 'multiplexing', 'panel_type', 'row_address_type',
'pixel_mapper_config', 'orientation']: 'pixel_mapper_config']:
if field in data: if field in data:
if field in ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'scan_mode', if field in ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'scan_mode',
'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz', 'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz',
@@ -1322,7 +1314,7 @@ def save_main_config():
return error_response( return error_response(
ErrorCode.CONFIG_SAVE_FAILED, ErrorCode.CONFIG_SAVE_FAILED,
"An error occurred; see logs for details", "An error occurred; see logs for details",
status_code=500, details=describe_exception(e) status_code=500
) )
@api_v3.route('/config/secrets', methods=['GET']) @api_v3.route('/config/secrets', methods=['GET'])
@@ -1336,7 +1328,7 @@ def get_secrets_config():
return jsonify({'status': 'success', 'data': config}) return jsonify({'status': 'success', 'data': config})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/config/raw/main', methods=['POST']) @api_v3.route('/config/raw/main', methods=['POST'])
def save_raw_main_config(): def save_raw_main_config():
@@ -1369,7 +1361,6 @@ def save_raw_main_config():
return error_response( return error_response(
ErrorCode.CONFIG_SAVE_FAILED, ErrorCode.CONFIG_SAVE_FAILED,
error_message, error_message,
details=describe_exception(e),
context={'config_path': e.config_path} if hasattr(e, 'config_path') and e.config_path else None, context={'config_path': e.config_path} if hasattr(e, 'config_path') and e.config_path else None,
status_code=500 status_code=500
@@ -1379,7 +1370,6 @@ def save_raw_main_config():
return error_response( return error_response(
ErrorCode.UNKNOWN_ERROR, ErrorCode.UNKNOWN_ERROR,
error_message, error_message,
details=describe_exception(e),
status_code=500 status_code=500
) )
@@ -1419,8 +1409,7 @@ def save_raw_secrets_config():
else: else:
error_message = 'An error occurred; see logs for details' error_message = 'An error occurred; see logs for details'
return jsonify({'status': 'error', 'message': error_message, return jsonify({'status': 'error', 'message': error_message}), 500
'details': describe_exception(e)}), 500
@api_v3.route('/system/status', methods=['GET']) @api_v3.route('/system/status', methods=['GET'])
def get_system_status(): def get_system_status():
@@ -1508,7 +1497,7 @@ def get_system_status():
return jsonify({'status': 'success', 'data': status}) return jsonify({'status': 'success', 'data': status})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/health', methods=['GET']) @api_v3.route('/health', methods=['GET'])
def get_health(): def get_health():
@@ -1607,11 +1596,9 @@ def get_health():
return jsonify({'status': 'success', 'data': health_status}) return jsonify({'status': 'success', 'data': health_status})
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'message': 'An error occurred; see logs for details',
'details': describe_exception(e),
'data': {'status': 'unhealthy'} 'data': {'status': 'unhealthy'}
}), 500 }), 500
@@ -2381,7 +2368,7 @@ def get_display_current():
return jsonify({'status': 'success', 'data': display_data}) return jsonify({'status': 'success', 'data': display_data})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/display/on-demand/status', methods=['GET']) @api_v3.route('/display/on-demand/status', methods=['GET'])
def get_on_demand_status(): def get_on_demand_status():
@@ -2405,7 +2392,7 @@ def get_on_demand_status():
}) })
except Exception as exc: except Exception as exc:
logger.error('Error in get_on_demand_status', exc_info=True) logger.error('Error in get_on_demand_status', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(exc)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/display/on-demand/start', methods=['POST']) @api_v3.route('/display/on-demand/start', methods=['POST'])
def start_on_demand_display(): def start_on_demand_display():
@@ -2508,7 +2495,7 @@ def start_on_demand_display():
return jsonify({'status': 'success', 'data': response_data}) return jsonify({'status': 'success', 'data': response_data})
except Exception as exc: except Exception as exc:
logger.error('Error in start_on_demand_display', exc_info=True) logger.error('Error in start_on_demand_display', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(exc)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/display/on-demand/stop', methods=['POST']) @api_v3.route('/display/on-demand/stop', methods=['POST'])
def stop_on_demand_display(): def stop_on_demand_display():
@@ -2544,7 +2531,7 @@ def stop_on_demand_display():
}) })
except Exception as exc: except Exception as exc:
logger.error('Error in stop_on_demand_display', exc_info=True) logger.error('Error in stop_on_demand_display', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(exc)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/installed', methods=['GET']) @api_v3.route('/plugins/installed', methods=['GET'])
def get_installed_plugins(): def get_installed_plugins():
@@ -2692,7 +2679,7 @@ def get_installed_plugins():
return jsonify({'status': 'success', 'data': {'plugins': plugins}}) return jsonify({'status': 'success', 'data': {'plugins': plugins}})
except Exception as e: except Exception as e:
logger.error('Error in get_installed_plugins', exc_info=True) logger.error('Error in get_installed_plugins', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
def _installed_plugin_ids(): def _installed_plugin_ids():
"""Best-effort list of installed plugin IDs for the web process. """Best-effort list of installed plugin IDs for the web process.
@@ -2758,7 +2745,7 @@ def get_plugin_health():
}) })
except Exception as e: except Exception as e:
logger.error('Error in get_plugin_health', exc_info=True) logger.error('Error in get_plugin_health', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/health/<plugin_id>', methods=['GET']) @api_v3.route('/plugins/health/<plugin_id>', methods=['GET'])
def get_plugin_health_single(plugin_id): def get_plugin_health_single(plugin_id):
@@ -2783,7 +2770,7 @@ def get_plugin_health_single(plugin_id):
}) })
except Exception as e: except Exception as e:
logger.error('Error in get_plugin_health_single', exc_info=True) logger.error('Error in get_plugin_health_single', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/health/<plugin_id>/reset', methods=['POST']) @api_v3.route('/plugins/health/<plugin_id>/reset', methods=['POST'])
def reset_plugin_health(plugin_id): def reset_plugin_health(plugin_id):
@@ -2808,7 +2795,7 @@ def reset_plugin_health(plugin_id):
}) })
except Exception as e: except Exception as e:
logger.error('Error in reset_plugin_health', exc_info=True) logger.error('Error in reset_plugin_health', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/metrics', methods=['GET']) @api_v3.route('/plugins/metrics', methods=['GET'])
def get_plugin_metrics(): def get_plugin_metrics():
@@ -2848,7 +2835,7 @@ def get_plugin_metrics():
}) })
except Exception as e: except Exception as e:
logger.error('Error in get_plugin_metrics', exc_info=True) logger.error('Error in get_plugin_metrics', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/metrics/<plugin_id>', methods=['GET']) @api_v3.route('/plugins/metrics/<plugin_id>', methods=['GET'])
def get_plugin_metrics_single(plugin_id): def get_plugin_metrics_single(plugin_id):
@@ -2873,7 +2860,7 @@ def get_plugin_metrics_single(plugin_id):
}) })
except Exception as e: except Exception as e:
logger.error('Error in get_plugin_metrics_single', exc_info=True) logger.error('Error in get_plugin_metrics_single', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/metrics/<plugin_id>/reset', methods=['POST']) @api_v3.route('/plugins/metrics/<plugin_id>/reset', methods=['POST'])
def reset_plugin_metrics(plugin_id): def reset_plugin_metrics(plugin_id):
@@ -2898,7 +2885,7 @@ def reset_plugin_metrics(plugin_id):
}) })
except Exception as e: except Exception as e:
logger.error('Error in reset_plugin_metrics', exc_info=True) logger.error('Error in reset_plugin_metrics', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/limits/<plugin_id>', methods=['GET', 'POST']) @api_v3.route('/plugins/limits/<plugin_id>', methods=['GET', 'POST'])
def manage_plugin_limits(plugin_id): def manage_plugin_limits(plugin_id):
@@ -2953,7 +2940,7 @@ def manage_plugin_limits(plugin_id):
}) })
except Exception as e: except Exception as e:
logger.error('Error in manage_plugin_limits', exc_info=True) logger.error('Error in manage_plugin_limits', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/toggle', methods=['POST']) @api_v3.route('/plugins/toggle', methods=['POST'])
def toggle_plugin(): def toggle_plugin():
@@ -3962,7 +3949,7 @@ def install_plugin():
except Exception as e: except Exception as e:
logger.error('Error in install_plugin', exc_info=True) logger.error('Error in install_plugin', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/install-from-url', methods=['POST']) @api_v3.route('/plugins/install-from-url', methods=['POST'])
def install_plugin_from_url(): def install_plugin_from_url():
@@ -4017,7 +4004,7 @@ def install_plugin_from_url():
except Exception as e: except Exception as e:
logger.error('Error in install_plugin_from_url', exc_info=True) logger.error('Error in install_plugin_from_url', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/registry-from-url', methods=['POST']) @api_v3.route('/plugins/registry-from-url', methods=['POST'])
def get_registry_from_url(): def get_registry_from_url():
@@ -4049,7 +4036,7 @@ def get_registry_from_url():
except Exception as e: except Exception as e:
logger.error('Error in get_registry_from_url', exc_info=True) logger.error('Error in get_registry_from_url', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/saved-repositories', methods=['GET']) @api_v3.route('/plugins/saved-repositories', methods=['GET'])
def get_saved_repositories(): def get_saved_repositories():
@@ -4062,7 +4049,7 @@ def get_saved_repositories():
return jsonify({'status': 'success', 'data': {'repositories': repositories}}) return jsonify({'status': 'success', 'data': {'repositories': repositories}})
except Exception as e: except Exception as e:
logger.error('Error in get_saved_repositories', exc_info=True) logger.error('Error in get_saved_repositories', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/saved-repositories', methods=['POST']) @api_v3.route('/plugins/saved-repositories', methods=['POST'])
def add_saved_repository(): def add_saved_repository():
@@ -4093,7 +4080,7 @@ def add_saved_repository():
}), 400 }), 400
except Exception as e: except Exception as e:
logger.error('Error in add_saved_repository', exc_info=True) logger.error('Error in add_saved_repository', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/saved-repositories', methods=['DELETE']) @api_v3.route('/plugins/saved-repositories', methods=['DELETE'])
def remove_saved_repository(): def remove_saved_repository():
@@ -4123,7 +4110,7 @@ def remove_saved_repository():
}), 404 }), 404
except Exception as e: except Exception as e:
logger.error('Error in remove_saved_repository', exc_info=True) logger.error('Error in remove_saved_repository', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/store/list', methods=['GET']) @api_v3.route('/plugins/store/list', methods=['GET'])
def list_plugin_store(): def list_plugin_store():
@@ -4176,7 +4163,7 @@ def list_plugin_store():
return jsonify({'status': 'success', 'data': {'plugins': formatted_plugins}}) return jsonify({'status': 'success', 'data': {'plugins': formatted_plugins}})
except Exception as e: except Exception as e:
logger.error('Error in list_plugin_store', exc_info=True) logger.error('Error in list_plugin_store', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/store/github-status', methods=['GET']) @api_v3.route('/plugins/store/github-status', methods=['GET'])
def get_github_auth_status(): def get_github_auth_status():
@@ -4227,7 +4214,7 @@ def get_github_auth_status():
}) })
except Exception as e: except Exception as e:
logger.error('Error in get_github_auth_status', exc_info=True) logger.error('Error in get_github_auth_status', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/store/refresh', methods=['POST']) @api_v3.route('/plugins/store/refresh', methods=['POST'])
def refresh_plugin_store(): def refresh_plugin_store():
@@ -4254,7 +4241,7 @@ def refresh_plugin_store():
}) })
except Exception as e: except Exception as e:
logger.error('Error in refresh_plugin_store', exc_info=True) logger.error('Error in refresh_plugin_store', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
def deep_merge(base_dict, update_dict): def deep_merge(base_dict, update_dict):
""" """
@@ -5776,7 +5763,7 @@ def get_plugin_schema():
return jsonify({'status': 'success', 'data': {'schema': default_schema}}) return jsonify({'status': 'success', 'data': {'schema': default_schema}})
except Exception as e: except Exception as e:
logger.error('Error in get_plugin_schema', exc_info=True) logger.error('Error in get_plugin_schema', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/skins', methods=['GET']) @api_v3.route('/skins', methods=['GET'])
def list_skins(): def list_skins():
@@ -5811,9 +5798,9 @@ def list_skins():
'has_preview': bool(preview and (skin_dir / preview).is_file()), 'has_preview': bool(preview and (skin_dir / preview).is_file()),
}) })
return jsonify({'status': 'success', 'data': {'skins': payload}}) return jsonify({'status': 'success', 'data': {'skins': payload}})
except Exception as e: except Exception:
logger.error('Error in list_skins', exc_info=True) logger.error('Error in list_skins', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/config/reset', methods=['POST']) @api_v3.route('/plugins/config/reset', methods=['POST'])
def reset_plugin_config(): def reset_plugin_config():
@@ -5893,7 +5880,7 @@ def reset_plugin_config():
}) })
except Exception as e: except Exception as e:
logger.error('Error in reset_plugin_config', exc_info=True) logger.error('Error in reset_plugin_config', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/action', methods=['POST']) @api_v3.route('/plugins/action', methods=['POST'])
def execute_plugin_action(): def execute_plugin_action():
@@ -6153,7 +6140,7 @@ sys.exit(proc.returncode)
logger.error("Error executing action step 1", exc_info=True) logger.error("Error executing action step 1", exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'details': describe_exception(e) 'message': 'An error occurred; see logs for details'
}), 500 }), 500
else: else:
# Simple script execution # Simple script execution
@@ -6203,7 +6190,7 @@ sys.exit(proc.returncode)
return jsonify({'status': 'error', 'message': 'Action timed out'}), 408 return jsonify({'status': 'error', 'message': 'Action timed out'}), 408
except Exception as e: except Exception as e:
logger.error('Error in execute_plugin_action', exc_info=True) logger.error('Error in execute_plugin_action', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/authenticate/spotify', methods=['POST']) @api_v3.route('/plugins/authenticate/spotify', methods=['POST'])
def authenticate_spotify(): def authenticate_spotify():
@@ -6336,12 +6323,12 @@ sys.exit(proc.returncode)
logger.error("Error getting Spotify auth URL", exc_info=True) logger.error("Error getting Spotify auth URL", exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'details': describe_exception(e) 'message': 'An error occurred; see logs for details'
}), 500 }), 500
except Exception as e: except Exception as e:
logger.error('Error in authenticate_spotify', exc_info=True) logger.error('Error in authenticate_spotify', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/authenticate/ytm', methods=['POST']) @api_v3.route('/plugins/authenticate/ytm', methods=['POST'])
def authenticate_ytm(): def authenticate_ytm():
@@ -6391,7 +6378,7 @@ def authenticate_ytm():
return jsonify({'status': 'error', 'message': 'Authentication timed out'}), 408 return jsonify({'status': 'error', 'message': 'Authentication timed out'}), 408
except Exception as e: except Exception as e:
logger.error('Error in authenticate_ytm', exc_info=True) logger.error('Error in authenticate_ytm', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/fonts/catalog', methods=['GET']) @api_v3.route('/fonts/catalog', methods=['GET'])
def get_fonts_catalog(): def get_fonts_catalog():
@@ -6486,10 +6473,7 @@ def get_fonts_catalog():
return jsonify({'status': 'success', 'data': {'catalog': catalog}}) return jsonify({'status': 'success', 'data': {'catalog': catalog}})
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True) return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
return jsonify({'status': 'error',
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)}), 500
@api_v3.route('/fonts/tokens', methods=['GET']) @api_v3.route('/fonts/tokens', methods=['GET'])
def get_font_tokens(): def get_font_tokens():
@@ -6508,7 +6492,7 @@ def get_font_tokens():
return jsonify({'status': 'success', 'data': {'tokens': tokens}}) return jsonify({'status': 'success', 'data': {'tokens': tokens}})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/fonts/overrides', methods=['GET']) @api_v3.route('/fonts/overrides', methods=['GET'])
def get_fonts_overrides(): def get_fonts_overrides():
@@ -6520,7 +6504,7 @@ def get_fonts_overrides():
return jsonify({'status': 'success', 'data': {'overrides': overrides}}) return jsonify({'status': 'success', 'data': {'overrides': overrides}})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/fonts/overrides', methods=['POST']) @api_v3.route('/fonts/overrides', methods=['POST'])
def save_fonts_overrides(): def save_fonts_overrides():
@@ -6534,7 +6518,7 @@ def save_fonts_overrides():
return jsonify({'status': 'success', 'message': 'Font overrides saved'}) return jsonify({'status': 'success', 'message': 'Font overrides saved'})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/fonts/overrides/<element_key>', methods=['DELETE']) @api_v3.route('/fonts/overrides/<element_key>', methods=['DELETE'])
def delete_font_override(element_key): def delete_font_override(element_key):
@@ -6544,7 +6528,7 @@ def delete_font_override(element_key):
return jsonify({'status': 'success', 'message': f'Font override for {element_key} deleted'}) return jsonify({'status': 'success', 'message': f'Font override for {element_key} deleted'})
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/fonts/upload', methods=['POST']) @api_v3.route('/fonts/upload', methods=['POST'])
def upload_font(): def upload_font():
@@ -6609,7 +6593,7 @@ def upload_font():
}) })
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/fonts/preview', methods=['GET']) @api_v3.route('/fonts/preview', methods=['GET'])
@@ -6754,7 +6738,7 @@ def get_font_preview() -> tuple[Response, int] | Response:
}) })
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/fonts/<font_family>', methods=['DELETE']) @api_v3.route('/fonts/<font_family>', methods=['DELETE'])
@@ -6842,7 +6826,7 @@ def delete_font(font_family: str) -> tuple[Response, int] | Response:
}) })
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/assets/upload', methods=['POST']) @api_v3.route('/plugins/assets/upload', methods=['POST'])
@@ -6990,7 +6974,7 @@ def upload_plugin_asset():
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/of-the-day/json/upload', methods=['POST']) @api_v3.route('/plugins/of-the-day/json/upload', methods=['POST'])
def upload_of_the_day_json(): def upload_of_the_day_json():
@@ -7140,7 +7124,7 @@ def upload_of_the_day_json():
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/of-the-day/json/delete', methods=['POST']) @api_v3.route('/plugins/of-the-day/json/delete', methods=['POST'])
def delete_of_the_day_json(): def delete_of_the_day_json():
@@ -7187,7 +7171,7 @@ def delete_of_the_day_json():
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/<plugin_id>/static/<path:file_path>', methods=['GET']) @api_v3.route('/plugins/<plugin_id>/static/<path:file_path>', methods=['GET'])
def serve_plugin_static(plugin_id, file_path): def serve_plugin_static(plugin_id, file_path):
@@ -7233,7 +7217,7 @@ def serve_plugin_static(plugin_id, file_path):
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/calendar/upload-credentials', methods=['POST']) @api_v3.route('/plugins/calendar/upload-credentials', methods=['POST'])
@@ -7315,228 +7299,7 @@ def upload_calendar_credentials():
except Exception as e: except Exception as e:
logger.error('Error in upload_calendar_credentials', exc_info=True) logger.error('Error in upload_calendar_credentials', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
# calendarList.list pages at 250 entries maximum. Ten pages is far past any
# real account and exists only so a malformed nextPageToken cannot spin here.
_CALENDAR_LIST_MAX_PAGES = 10
def _calendar_plugin_dir() -> Optional[Path]:
"""Where the calendar plugin is installed, or None if it is not."""
if api_v3.plugin_manager:
plugin_dir = api_v3.plugin_manager.get_plugin_directory('calendar')
else:
plugin_dir = PROJECT_ROOT / 'plugins' / 'calendar'
if not plugin_dir:
return None
plugin_dir = Path(plugin_dir)
return plugin_dir if plugin_dir.exists() else None
def _run_calendar_registration(plugin_dir: Path, stdin_payload: str):
"""Run the plugin's OAuth script and return the JSON object it prints.
The script decides between web and terminal mode by whether stdin is a
tty, so it must be given a pipe. It emits one JSON object on stdout; the
last parsable line is taken, because an import warning or a library's
stderr redirection can land in front of it.
Returns (payload, error_message). Exactly one is None.
"""
script = plugin_dir / 'calendar_registration.py'
if not script.exists():
return None, 'Authentication script not found in the calendar plugin'
try:
result = subprocess.run( # nosec B603 - fixed script path inside the plugin dir
[sys.executable, str(script)],
input=stdin_payload,
capture_output=True,
text=True,
timeout=120,
cwd=str(plugin_dir),
)
except subprocess.TimeoutExpired:
return None, 'Authentication timed out after 120s'
except OSError as e:
logger.error('Could not run calendar_registration.py', exc_info=True)
return None, 'Could not run the authentication script: %s' % describe_exception(e)
for line in reversed((result.stdout or '').splitlines()):
line = line.strip()
if not line:
continue
try:
payload = json.loads(line)
except json.JSONDecodeError:
continue
if isinstance(payload, dict):
return payload, None
raw = (result.stderr or result.stdout or '').strip()
# The unredacted text goes to the log, where it is worth having in full.
# What comes back over HTTP is redacted: this is a script that handles
# OAuth client secrets, and its stderr can quote them.
if raw:
logger.error('calendar_registration.py failed (exit %s): %s',
result.returncode, raw)
return None, 'Authentication script produced no result%s' % (
': %s' % redact_text(raw) if raw else '')
@api_v3.route('/plugins/calendar/authenticate', methods=['POST'])
def authenticate_calendar():
"""Google OAuth for the calendar plugin, in the two steps it requires.
Step 1 (no body) returns the consent URL to open. Step 2 posts back the
URL Google redirected to -- it fails to load, because the redirect points
at a loopback address nothing is listening on, but the address bar carries
the authorization code -- and the script exchanges it for a token.
Two calls rather than one because the user has to visit Google in between.
The script persists the PKCE verifier from step 1 for step 2 to reuse; the
exchange fails with "Missing code verifier" otherwise.
"""
try:
plugin_dir = _calendar_plugin_dir()
if plugin_dir is None:
return jsonify({
'status': 'error',
'message': 'The calendar plugin is not installed'
}), 404
if not (plugin_dir / 'credentials.json').exists():
return jsonify({
'status': 'error',
'message': ('No credentials.json yet. Upload your Google OAuth '
'client file first (Step 1).')
}), 400
data = request.get_json(silent=True) or {}
redirect_url = (data.get('redirect_url') or data.get('code') or '').strip()
payload, error = _run_calendar_registration(plugin_dir, redirect_url)
if error:
return jsonify({'status': 'error', 'message': error}), 500
if payload.get('status') != 'success':
# The script's own diagnosis is more useful than anything that
# could be reconstructed here -- but it interpolates exceptions
# into its messages, so it reaches the client redacted and the
# original goes to the log.
logger.error('calendar authentication failed: %s', payload)
safe = dict(payload)
safe['message'] = redact_text(str(payload.get('message', '')
or 'Authentication failed'))
return jsonify(safe), 400
return jsonify(payload)
except Exception as e:
logger.error('Error in authenticate_calendar', exc_info=True)
return jsonify({'status': 'error',
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)}), 500
@api_v3.route('/plugins/calendar/list-calendars', methods=['GET'])
def list_calendar_calendars():
"""The calendars this account can see, for the config picker.
Reads the token the OAuth flow wrote rather than shelling out again: the
picker is used interactively and a subprocess per click is slower than the
API call it would be wrapping.
"""
try:
plugin_dir = _calendar_plugin_dir()
if plugin_dir is None:
return jsonify({
'status': 'error',
'message': 'The calendar plugin is not installed'
}), 404
token_file = plugin_dir / 'token.pickle'
if not token_file.exists():
return jsonify({
'status': 'error',
'message': ('Not authenticated with Google yet. Complete Step 2 '
'first, then load your calendars.')
}), 400
try:
import pickle
from google.auth.transport.requests import Request as GoogleRequest
from googleapiclient.discovery import build as build_google_service
except ImportError as e:
return jsonify({
'status': 'error',
# The name of the missing module is the whole diagnosis, but it
# arrives as an exception, so it goes through the redactor like
# any other -- an ImportError can quote a path.
'message': ('The Google API libraries are not installed. Install '
"the calendar plugin's requirements.txt. (%s)"
% describe_exception(e))
}), 500
with open(token_file, 'rb') as handle:
# Written only by this plugin's own OAuth flow, into its own
# directory, and read here exactly as the plugin itself reads it.
creds = pickle.load(handle) # nosec B301 - locally generated token
if creds and creds.expired and creds.refresh_token:
creds.refresh(GoogleRequest())
with open(token_file, 'wb') as handle:
pickle.dump(creds, handle)
os.chmod(token_file, 0o600)
if not creds or not creds.valid:
return jsonify({
'status': 'error',
'message': ('Stored Google credentials are no longer valid. '
'Run Step 2 again to re-authenticate.')
}), 400
service = build_google_service('calendar', 'v3', credentials=creds)
# calendarList.list returns 100 entries per page by default and caps at
# 250, handing back a nextPageToken when there are more. Taking only
# the first page would silently hide calendars from the picker, and the
# user would have no way to tell the list was truncated.
entries = []
page_token = None
for _ in range(_CALENDAR_LIST_MAX_PAGES):
response = service.calendarList().list(
maxResults=250, pageToken=page_token).execute()
entries.extend(response.get('items', []))
page_token = response.get('nextPageToken')
if not page_token:
break
else:
# 2500 calendars in, something is wrong with the account or the
# token is looping; show what was collected rather than spin.
logger.warning(
'calendarList paging stopped at %d pages with more remaining',
_CALENDAR_LIST_MAX_PAGES)
calendars = [{
'id': entry.get('id'),
# The picker labels each row with summary and falls back to the id
# only in its own display, so send something either way.
'summary': entry.get('summary') or entry.get('id'),
'primary': bool(entry.get('primary', False)),
} for entry in entries if entry.get('id')]
# Primary first, then alphabetically: the list is usually short but the
# one the user wants is almost always their own calendar.
calendars.sort(key=lambda c: (not c['primary'], c['summary'].lower()))
return jsonify({'status': 'success', 'calendars': calendars})
except Exception as e:
logger.error('Error in list_calendar_calendars', exc_info=True)
return jsonify({'status': 'error',
'message': 'An error occurred; see logs for details',
'details': describe_exception(e)}), 500
@api_v3.route('/plugins/assets/delete', methods=['POST']) @api_v3.route('/plugins/assets/delete', methods=['POST'])
def delete_plugin_asset(): def delete_plugin_asset():
@@ -7579,7 +7342,7 @@ def delete_plugin_asset():
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/assets/list', methods=['GET']) @api_v3.route('/plugins/assets/list', methods=['GET'])
def list_plugin_assets(): def list_plugin_assets():
@@ -7607,7 +7370,7 @@ def list_plugin_assets():
except Exception as e: except Exception as e:
logger.error('Unhandled exception', exc_info=True) logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/display/current-status', methods=['GET']) @api_v3.route('/display/current-status', methods=['GET'])
def get_current_display_status(): def get_current_display_status():
@@ -7628,9 +7391,9 @@ def get_current_display_status():
'last_updated': None, 'last_updated': None,
} }
return jsonify({'status': 'success', 'data': state}) return jsonify({'status': 'success', 'data': state})
except Exception as e: except Exception:
logger.error('Error in get_current_display_status', exc_info=True) logger.error('Error in get_current_display_status', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/logs', methods=['GET']) @api_v3.route('/logs', methods=['GET'])
def get_logs(): def get_logs():
@@ -7669,11 +7432,9 @@ def get_logs():
'message': 'Timeout while fetching logs' 'message': 'Timeout while fetching logs'
}), 500 }), 500
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'message': 'An error occurred; see logs for details'
'details': describe_exception(e)
}), 500 }), 500
# Multi-Display Sync Endpoints # Multi-Display Sync Endpoints
@@ -7738,11 +7499,9 @@ def get_wifi_status():
} }
}) })
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'message': 'An error occurred; see logs for details'
'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/scan', methods=['GET']) @api_v3.route('/wifi/scan', methods=['GET'])
@@ -7864,7 +7623,7 @@ def connect_wifi():
logger.error("Error connecting to WiFi", exc_info=True) logger.error("Error connecting to WiFi", exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'details': describe_exception(e) 'message': 'An error occurred; see logs for details'
}), 500 }), 500
@api_v3.route('/wifi/disconnect', methods=['POST']) @api_v3.route('/wifi/disconnect', methods=['POST'])
@@ -7890,7 +7649,7 @@ def disconnect_wifi():
logger.error("Error disconnecting from WiFi", exc_info=True) logger.error("Error disconnecting from WiFi", exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'details': describe_exception(e) 'message': 'An error occurred; see logs for details'
}), 500 }), 500
@api_v3.route('/wifi/ap/enable', methods=['POST']) @api_v3.route('/wifi/ap/enable', methods=['POST'])
@@ -7915,11 +7674,9 @@ def enable_ap_mode():
'message': message 'message': message
}), 400 }), 400
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'message': 'An error occurred; see logs for details'
'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/ap/disable', methods=['POST']) @api_v3.route('/wifi/ap/disable', methods=['POST'])
@@ -7942,11 +7699,9 @@ def disable_ap_mode():
'message': message 'message': message
}), 400 }), 400
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'message': 'An error occurred; see logs for details'
'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/ap/auto-enable', methods=['GET']) @api_v3.route('/wifi/ap/auto-enable', methods=['GET'])
@@ -7965,11 +7720,9 @@ def get_auto_enable_ap_mode():
} }
}) })
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'message': 'An error occurred; see logs for details'
'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/ap/auto-enable', methods=['POST']) @api_v3.route('/wifi/ap/auto-enable', methods=['POST'])
@@ -7999,11 +7752,9 @@ def set_auto_enable_ap_mode():
} }
}) })
except Exception as e: except Exception as e:
logger.error("%s failed", request.path, exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'message': 'An error occurred; see logs for details'
'details': describe_exception(e)
}), 500 }), 500
@api_v3.route('/wifi/radio', methods=['GET']) @api_v3.route('/wifi/radio', methods=['GET'])
@@ -8023,7 +7774,7 @@ def get_wifi_radio():
logger.error("Error getting WiFi radio state", exc_info=True) logger.error("Error getting WiFi radio state", exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'details': describe_exception(e) 'message': 'An error occurred; see logs for details'
}), 500 }), 500
@api_v3.route('/wifi/radio', methods=['POST']) @api_v3.route('/wifi/radio', methods=['POST'])
@@ -8071,7 +7822,7 @@ def set_wifi_radio():
logger.error("Error setting WiFi radio state", exc_info=True) logger.error("Error setting WiFi radio state", exc_info=True)
return jsonify({ return jsonify({
'status': 'error', 'status': 'error',
'message': 'An error occurred; see logs for details', 'details': describe_exception(e) 'message': 'An error occurred; see logs for details'
}), 500 }), 500
@api_v3.route('/cache/list', methods=['GET']) @api_v3.route('/cache/list', methods=['GET'])
@@ -8096,7 +7847,7 @@ def list_cache_files():
}) })
except Exception as e: except Exception as e:
logger.error('Error in list_cache_files', exc_info=True) logger.error('Error in list_cache_files', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/cache/delete', methods=['POST']) @api_v3.route('/cache/delete', methods=['POST'])
def delete_cache_file(): def delete_cache_file():
@@ -8122,7 +7873,7 @@ def delete_cache_file():
}) })
except Exception as e: except Exception as e:
logger.error('Error in delete_cache_file', exc_info=True) logger.error('Error in delete_cache_file', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details', 'details': describe_exception(e)}), 500 return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
# ============================================================================= # =============================================================================
@@ -1,196 +0,0 @@
/**
* Google OAuth Widget
*
* Step 2 of the calendar plugin's setup, between uploading the OAuth client
* file and picking calendars. Google will not let a headless device complete
* consent on its own, so the flow is necessarily two calls with a human in
* between:
*
* 1. POST /api/v3/plugins/calendar/authenticate with no body
* -> { auth_url } to open in a browser
* 2. the browser lands on a loopback address that fails to load; its URL
* carries the authorization code. POST it back as redirect_url
* -> the server exchanges it and writes token.pickle
*
* The failed page in step 2 is expected and is worth saying out loud, because
* it looks exactly like something went wrong.
*
* @module GoogleOAuthWidget
*/
(function () {
'use strict';
if (typeof window.LEDMatrixWidgets === 'undefined') {
console.error('[GoogleOAuthWidget] LEDMatrixWidgets registry not found. Load registry.js first.');
return;
}
const ENDPOINT = '/api/v3/plugins/calendar/authenticate';
window.LEDMatrixWidgets.register('google-oauth', {
name: 'Google OAuth Widget',
version: '1.0.0',
/**
* @param {HTMLElement} container
* @param {Object} config - schema config (unused)
* @param {*} value - unused; this widget stores nothing
* @param {Object} options - { fieldId, pluginId, name }
*/
render: function (container, config, value, options) {
const fieldId = options.fieldId;
// Nothing is stored in config by this step -- the result is
// token.pickle on the device -- but the form still expects a field.
const hidden = document.createElement('input');
hidden.type = 'hidden';
hidden.id = fieldId + '_hidden';
hidden.name = options.name;
hidden.value = value || '';
const startBtn = document.createElement('button');
startBtn.type = 'button';
startBtn.className = 'px-3 py-1.5 text-sm rounded-md bg-blue-600 hover:bg-blue-700 text-white';
startBtn.innerHTML = '<i class="fas fa-key"></i> Connect Google Account';
const status = document.createElement('p');
status.className = 'text-xs text-gray-400 mt-2';
// Every message this widget gives -- the consent link is ready,
// the exchange failed -- arrives here after an async call, so a
// screen reader is told nothing unless it is a live region.
status.setAttribute('role', 'status');
status.setAttribute('aria-live', 'polite');
const step2 = document.createElement('div');
step2.className = 'mt-3 hidden';
const link = document.createElement('a');
link.target = '_blank';
link.rel = 'noopener noreferrer';
link.className = 'text-blue-400 underline text-sm break-all';
link.textContent = 'Open the Google consent screen';
// Deliberately loud. After consent the browser is redirected to a
// loopback address nothing is listening on, so it lands on a
// browser error page -- which reads as a failure at exactly the
// moment the user has to act on it. Said quietly in grey it gets
// missed, and the flow looks broken when it is working.
const hint = document.createElement('div');
hint.className =
'mt-3 p-3 rounded-md border border-amber-500/60 bg-amber-500/10';
hint.innerHTML =
'<p class="text-sm text-amber-300 font-semibold">'
+ '<i class="fas fa-triangle-exclamation"></i> '
+ 'The next page will fail to load. That is expected.</p>'
+ '<p class="text-xs text-amber-200/90 mt-1">'
+ 'After you approve access, Google sends your browser to '
+ '<code>127.0.0.1</code>, where nothing is running \u2014 so you will see '
+ '"This site can\u2019t be reached" or similar. Nothing has gone wrong. '
+ 'Copy the <strong>entire address</strong> out of the address bar '
+ '(it contains <code>?code=...</code>) and paste it in the box below.</p>';
const codeInputId = fieldId + '_redirect_url';
const codeLabel = document.createElement('label');
codeLabel.className = 'block text-xs text-gray-300 mt-3';
codeLabel.textContent = 'Paste the address from that failed page here:';
// The label was visible but not associated, so the input still had
// no accessible name -- a placeholder is not one, and it vanishes
// on focus, which is exactly when the value is being pasted.
codeLabel.setAttribute('for', codeInputId);
const codeInput = document.createElement('input');
codeInput.type = 'text';
codeInput.id = codeInputId;
codeInput.placeholder = 'http://127.0.0.1/?code=...';
codeInput.className =
'mt-2 block w-full px-3 py-2 text-sm border border-gray-600 '
+ 'rounded-md bg-gray-800 text-gray-100';
const finishBtn = document.createElement('button');
finishBtn.type = 'button';
finishBtn.className = 'mt-2 px-3 py-1.5 text-sm rounded-md bg-green-600 hover:bg-green-700 text-white';
finishBtn.innerHTML = '<i class="fas fa-check"></i> Finish Authentication';
function say(message, kind) {
status.textContent = message;
status.className = 'text-xs mt-2 ' + (
kind === 'error' ? 'text-red-400'
: kind === 'success' ? 'text-green-400'
: 'text-gray-400');
}
function post(body) {
return fetch(ENDPOINT, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body || {})
}).then(function (r) {
return r.json().catch(function () {
// A non-JSON body here means the request never reached
// the handler -- worth saying so rather than "undefined".
return { status: 'error', message: 'Server returned ' + r.status };
});
});
}
startBtn.addEventListener('click', function () {
startBtn.disabled = true;
say('Requesting a consent link...');
post({}).then(function (data) {
startBtn.disabled = false;
if (data.status !== 'success' || !data.auth_url) {
say(data.message || 'Could not start authentication.', 'error');
return;
}
link.href = data.auth_url;
step2.classList.remove('hidden');
say(data.message || 'Open the link, approve, then paste the address back.');
}).catch(function (err) {
startBtn.disabled = false;
say('Request failed: ' + err.message, 'error');
});
});
finishBtn.addEventListener('click', function () {
const pasted = codeInput.value.trim();
if (!pasted) {
say('Paste the address your browser was redirected to.', 'error');
return;
}
finishBtn.disabled = true;
say('Exchanging the code with Google...');
post({ redirect_url: pasted }).then(function (data) {
finishBtn.disabled = false;
if (data.status !== 'success') {
say(data.message || 'Authentication failed.', 'error');
return;
}
say(data.message || 'Authenticated.', 'success');
step2.classList.add('hidden');
codeInput.value = '';
}).catch(function (err) {
finishBtn.disabled = false;
say('Request failed: ' + err.message, 'error');
});
});
step2.appendChild(link);
step2.appendChild(hint);
step2.appendChild(codeLabel);
step2.appendChild(codeInput);
step2.appendChild(finishBtn);
container.appendChild(hidden);
container.appendChild(startBtn);
container.appendChild(status);
container.appendChild(step2);
},
getValue: function (fieldId) {
const hidden = document.getElementById(fieldId + '_hidden');
return hidden ? hidden.value : '';
}
});
})();
-1
View File
@@ -987,7 +987,6 @@
<script src="{{ url_for('static', filename='v3/js/widgets/custom-feeds.js') }}" defer></script> <script src="{{ url_for('static', filename='v3/js/widgets/custom-feeds.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/array-table.js') }}" defer></script> <script src="{{ url_for('static', filename='v3/js/widgets/array-table.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/google-calendar-picker.js') }}" defer></script> <script src="{{ url_for('static', filename='v3/js/widgets/google-calendar-picker.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/google-oauth.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/day-selector.js') }}" defer></script> <script src="{{ url_for('static', filename='v3/js/widgets/day-selector.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/time-range.js') }}" defer></script> <script src="{{ url_for('static', filename='v3/js/widgets/time-range.js') }}" defer></script>
<script src="{{ url_for('static', filename='v3/js/widgets/time-picker.js') }}" defer></script> <script src="{{ url_for('static', filename='v3/js/widgets/time-picker.js') }}" defer></script>
@@ -117,14 +117,6 @@
</select> </select>
</div> </div>
<div class="form-group" id="setting-display-orientation" data-setting-key="display.hardware.orientation">
<label for="orientation" class="block text-sm font-medium text-gray-700">Panel Orientation{{ ui.help_tip('Rotates the rendered image to match how the panel is physically mounted.\nUse "Upside Down" if you flipped the panel 180° to move the Raspberry Pi / wiring to a more convenient side.', 'Panel Orientation') }}</label>
<select id="orientation" name="orientation" class="form-control">
<option value="normal" {% if main_config.display.hardware.get('orientation', 'normal') == "normal" %}selected{% endif %}>Normal</option>
<option value="180" {% if main_config.display.hardware.get('orientation', 'normal') == "180" %}selected{% endif %}>Upside Down (180°)</option>
</select>
</div>
<div class="form-group" id="setting-display-led_rgb_sequence" data-setting-key="display.hardware.led_rgb_sequence"> <div class="form-group" id="setting-display-led_rgb_sequence" data-setting-key="display.hardware.led_rgb_sequence">
<label for="led_rgb_sequence" class="block text-sm font-medium text-gray-700">LED RGB Sequence{{ ui.help_tip('Order the panel expects color channels in.\nChange this only if reds/greens/blues look swapped. Default: RGB.', 'LED RGB Sequence') }}</label> <label for="led_rgb_sequence" class="block text-sm font-medium text-gray-700">LED RGB Sequence{{ ui.help_tip('Order the panel expects color channels in.\nChange this only if reds/greens/blues look swapped. Default: RGB.', 'LED RGB Sequence') }}</label>
<select id="led_rgb_sequence" name="led_rgb_sequence" class="form-control"> <select id="led_rgb_sequence" name="led_rgb_sequence" class="form-control">
@@ -63,13 +63,13 @@
<!-- Getting Started checklist: non-gating, dismissible (localStorage), items <!-- Getting Started checklist: non-gating, dismissible (localStorage), items
auto-check from existing config/endpoints — no new persisted state. auto-check from existing config/endpoints — no new persisted state.
The timezone step is verified against the browser's own zone rather than Known heuristic limits (acceptable, disclosed): values left at legitimate
compared to the shipped default; see the data-check="timezone" block below defaults (e.g. a user actually in Tampa) read as "not done". -->
for why. -->
{% set _hw = main_config.display.hardware if main_config and main_config.display else {} %} {% set _hw = main_config.display.hardware if main_config and main_config.display else {} %}
{% set _hw_done = (_hw.rows or 0) > 0 and (_hw.cols or 0) > 0 and (_hw.chain_length or 0) > 0 %} {% set _hw_done = (_hw.rows or 0) > 0 and (_hw.cols or 0) > 0 and (_hw.chain_length or 0) > 0 %}
{% set _loc = main_config.location if main_config and main_config.location else {} %} {% set _loc = main_config.location if main_config and main_config.location else {} %}
{% set _tz = (main_config.timezone if main_config else '') or '' %} {% set _loc_done = (main_config.timezone and main_config.timezone != 'America/New_York')
or (_loc.city and _loc.city != 'Tampa') %}
<div id="getting-started-card" class="bg-blue-50 border border-blue-200 rounded-lg p-4 mb-4" style="display:none" role="region" aria-label="Getting started checklist"> <div id="getting-started-card" class="bg-blue-50 border border-blue-200 rounded-lg p-4 mb-4" style="display:none" role="region" aria-label="Getting started checklist">
<div class="flex items-start justify-between"> <div class="flex items-start justify-between">
<div class="flex-1"> <div class="flex-1">
@@ -78,8 +78,8 @@
<ul class="space-y-1 text-sm" id="getting-started-items"> <ul class="space-y-1 text-sm" id="getting-started-items">
<li><button type="button" class="gs-item text-left w-full" data-done="{{ '1' if _hw_done else '0' }}" data-tab="display"> <li><button type="button" class="gs-item text-left w-full" data-done="{{ '1' if _hw_done else '0' }}" data-tab="display">
<i class="far fa-square mr-2"></i>Set your panel size (Display tab)</button></li> <i class="far fa-square mr-2"></i>Set your panel size (Display tab)</button></li>
<li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="timezone" data-tz="{{ _tz }}" data-tab="general"> <li><button type="button" class="gs-item text-left w-full" data-done="{{ '1' if _loc_done else '0' }}" data-tab="general">
<i class="far fa-square mr-2"></i>Set your timezone{% if _tz %} — currently {{ _tz }}{% if _loc.city %}, {{ _loc.city }}{% endif %}{% endif %} (General tab)<span data-gs-tz-note class="text-xs"></span></button></li> <i class="far fa-square mr-2"></i>Set your timezone and location (General tab)</button></li>
<li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="installed" data-tab="plugins"> <li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="installed" data-tab="plugins">
<i class="far fa-square mr-2"></i>Install a plugin from the Plugin Store</button></li> <i class="far fa-square mr-2"></i>Install a plugin from the Plugin Store</button></li>
<li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="enabled" data-tab="plugins"> <li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="enabled" data-tab="plugins">
@@ -165,91 +165,6 @@
}); });
maybeAutoHide(); maybeAutoHide();
// Timezone: verified against the browser's own zone.
//
// This step used to tick when the saved timezone differed from the value
// config.template.json ships (America/New_York), with the saved city
// OR-ed in. Two things were wrong with that. "Differs from the default"
// answers "did somebody edit this?", but what the checklist needs to know
// is whether the value is RIGHT — so anyone who genuinely lives in the
// default zone could never satisfy it and the card nagged forever. And
// the city has no bearing on whether the timezone is set: because the two
// were OR-ed, saving a city ticked the step off with the timezone still
// wrong, which is the direction that actually breaks displays (event
// times render in the wrong zone).
//
// The browser already knows its zone, so compare against that: no new
// persisted state, no network, and it catches the reverse case too — a
// panel still set to the old zone after a move now stays unticked, where
// the old test ticked it the moment the value stopped being the default.
function sameZone(a, b) {
if (a === b) return true;
// Compare the wall-clock time each zone yields, not the identifiers:
// aliases (Asia/Calcutta vs Asia/Kolkata, Europe/Kiev vs Europe/Kyiv)
// name one zone and must not read as a mismatch.
//
// Sampled at three instants, all of which have to agree. Checking only
// now is not enough: America/New_York and America/Lima hold the same
// offset all winter, so a panel set to the wrong one of those would
// tick in January and then run an hour off from March. Mid-January and
// mid-July sit either side of DST in both hemispheres, so only zones
// that agree year-round match -- while Toronto still matches New York,
// which is right, since either renders the same times.
try {
var now = new Date();
var year = now.getUTCFullYear();
var instants = [now,
new Date(Date.UTC(year, 0, 15, 12)),
new Date(Date.UTC(year, 6, 15, 12))];
var stamp = function (tz, at) {
// Explicit numeric fields rather than dateStyle/timeStyle:
// those are late additions to Intl (Firefox shipped them in
// 91), and an implementation that does not know them ignores
// them and formats the date alone. That would compare
// New York, Chicago and Madrid as equal and tick the step for
// a timezone that is plainly wrong -- the exact failure this
// check exists to catch. These options have been in Intl
// since ECMA-402 v1.
return new Intl.DateTimeFormat('en-US', {
timeZone: tz, year: 'numeric', month: '2-digit',
day: '2-digit', hour: '2-digit', minute: '2-digit',
hour12: false
}).format(at);
};
for (var i = 0; i < instants.length; i++) {
if (stamp(a, instants[i]) !== stamp(b, instants[i])) {
return false;
}
}
return true;
} catch (e) {
// An unparseable zone in the config is worth surfacing, not hiding.
return false;
}
}
(function () {
var tzBtn = card.querySelector('[data-check="timezone"]');
if (!tzBtn) return;
var configured = tzBtn.dataset.tz || '';
if (!configured) return; // nothing saved yet: leave it open
var local = '';
try {
local = (Intl.DateTimeFormat().resolvedOptions().timeZone) || '';
} catch (e) {
return; // no Intl: leave it to the manual tick
}
if (!local) return;
if (sameZone(configured, local)) {
markDone(tzBtn);
return;
}
// Unticked on its own says "wrong" without saying why; name the zone
// the browser is in so the step is actionable.
var note = tzBtn.querySelector('[data-gs-tz-note]');
if (note) note.textContent = ' — this browser is in ' + local;
}());
// Plugin-derived states from the existing installed-plugins endpoint. // Plugin-derived states from the existing installed-plugins endpoint.
fetch('/api/v3/plugins/installed') fetch('/api/v3/plugins/installed')
.then(function (r) { return r.json(); }) .then(function (r) { return r.json(); })
@@ -815,7 +815,7 @@
<i class="fas fa-info-circle mr-1"></i> <i class="fas fa-info-circle mr-1"></i>
Changes in the file manager save immediately — no need to click Save Configuration. Changes in the file manager save immediately — no need to click Save Configuration.
</p> </p>
{% elif str_widget in ['text-input', 'textarea', 'select-dropdown', 'toggle-switch', 'radio-group', 'date-picker', 'time-picker', 'slider', 'color-picker', 'email-input', 'url-input', 'password-input', 'font-selector', 'file-upload-single', 'plugin-file-manager', 'google-oauth'] %} {% elif str_widget in ['text-input', 'textarea', 'select-dropdown', 'toggle-switch', 'radio-group', 'date-picker', 'time-picker', 'slider', 'color-picker', 'email-input', 'url-input', 'password-input', 'font-selector', 'file-upload-single', 'plugin-file-manager'] %}
{# Render widget container #} {# Render widget container #}
<div id="{{ field_id }}_container" class="{{ str_widget }}-container"></div> <div id="{{ field_id }}_container" class="{{ str_widget }}-container"></div>
<script> <script>