Files
LEDMatrix/plugin-repos/starlark-apps/manager.py
T
ChuckandClaude Opus 5 a0d3e64099 fix: ten defects found validating the whole plugin fleet on hardware (#534)
* fix(core): register tom_thumb, accept frame_hold in the test double, wire api_v3's managers

Three independent fixes found while validating every plugin on a 256x64 rig.

FontManager never registered tom_thumb even though assets/fonts/tom-thumb.bdf
ships with the core, so every plugin offering it logged "Font family
'tom_thumb' not found" (16 warnings per countdown render) and had to carry a
private loader to use a bundled font. Closes #524.

VisualTestDisplayManager.set_scrolling_state() lacked the frame_hold parameter
that DisplayManager gained, so any plugin passing it died with TypeError at
render time and failed every size. Nine plugins now make that call;
ledmatrix-stocks and ledmatrix-leaderboard were failing outright and the other
seven only passed because their scroll path was unreachable without data.
Closes #525.

api_v3 declared module-level config_manager/plugin_manager = None that nothing
ever assigned -- app.py sets the blueprint attributes, which the other 150+
call sites use. Three sites read the decoys, so /health reported the config
unreadable and the plugin system uninitialised (making "degraded" permanent and
unreachable-by-design) and /display/current fell back to a hardcoded 128x64 on
every rig. The decoys are removed rather than assigned, so a bare name is now a
NameError at test time instead of a silent None. The same function's first-call
uptime was computed from two separate clock reads and came out negative.
Closes #529.

Verified on the rig: both previously-failing plugins render, the tom_thumb
warnings are gone, /health reports "healthy" with all three checks passing, and
/display/current reports the real 256x64.

* fix(core): unique snapshot temp name, honour on-demand requests, skip empty starlark

The preview snapshot wrote through a fixed "<snapshot>.tmp". /tmp is
world-writable and sticky, and the display service runs as a different user
from the tooling, so a leftover temp owned by anyone else became unopenable
even by root -- fs.protected_regular refuses O_CREAT on a foreign file in a
sticky directory. The preview and the health check's liveness proxy then froze
until someone deleted the file by hand; on the test rig that meant 23 hours of
a healthy display reporting "hardware: stale". Now uses tempfile.mkstemp with
cleanup on failure, matching the hardware-status write a few hundred lines
above. Closes #528.

_poll_on_demand_requests read its mailbox with max_age=3600, and get() defaults
the in-memory TTL to max_age -- so the first request was pinned in memory for an
hour and every later poll returned that stale copy. No second on-demand request
was honoured until the service restarted, while the API kept returning 200.
get() already documents memory_ttl=0 for exactly this cross-process case.
The consumed request is also now deleted: leaving it on disk meant a restart
replayed the previous request, activated it, and ignored the one the caller had
just made. Closes #530.

starlark-apps returned None from display() when it has no app to show, which is
the state of every install without Pixlet and of a fresh one before any app is
added. The controller only skips on a boolean False, so that held a black panel
for the full display_duration instead of rotating on. Closes #456 (core side).

Verified on the rig: two consecutive on-demand requests with no restart between
them are both activated, where the second was previously dropped in silence.

* perf(harness): share one cache across a plugin's renders

_instantiate built a fresh MockCacheManager for every (size, mode), and that
mock is a per-instance in-memory dict, so each render was a cold start. A plugin
that fetches per game or per player re-fetched everything N times over --
baseball-scoreboard at one size took 840s for nine renders where the arithmetic
said ~72s, and at eight sizes it exceeded a 900s timeout.

The second and later renders also never exercised the cache-hit path, which is
what a running rig executes almost all of the time, so a caching regression
could not be caught here.

The cache is now built once per render_plugin_matrix call and threaded down.
The display manager stays per-render -- the bounds checking depends on that --
so only fetched data is shared.

Measured on the rig, same render counts and same goldens:
  tide-display         2s -> 1s   (32 renders)
  cricket-scoreboard  10s -> 3s   (24 renders)
No pass/fail change across tide-display, cricket-scoreboard, clock-simple,
geochron, christmas-countdown, of-the-day, web-ui-info and incoming-packages.

Closes #533.

* fix(scripts): run standalone plugin tests instead of collecting nothing

run_plugin_tests.py discovered every plugin test file and handed the lot to
pytest. Most plugin tests are standalone scripts -- module-level main() plus an
`if __name__ == "__main__"` guard, signalling through an exit code -- and pytest
collects zero items from those. The run printed how many files it had *found*,
then "no tests ran", and exited without executing any of them. On a rig with all
44 first-party plugins that is 151 of 248 files.

Files are now classified and each kind runs under the right runner: pytest for
real test modules, subprocess for scripts, honouring the 0 pass / 2 skip / 1
fail convention ledmatrix-plugins' own runner established (a script that wants a
tty or an LED matrix is a skip, not a regression).

Before:
    $ python3 scripts/run_plugin_tests.py -p countdown -d ~/LEDMatrix/plugin-repos
    Found 1 test file(s)
    collected 0 items
    no tests ran in 0.31s                      rc=0

After:
    Found 1 test file(s) -- 0 collectable, 1 standalone script(s)
    1 passed, 0 skipped, 0 failed (scripts)    rc=0

Verified across three shapes: countdown (1 script), jellyfin-now-playing and
pomodoro-timer (pytest only, 16 and 42 tests), and ledmatrix-flights (11 files
split 4 collectable / 7 scripts, all seven of which had never run).

Closes #532.

Running the flights scripts for the first time also surfaced four genuinely
failing tests there, hidden by the mirror-image bug in the plugins repo's own
runner -- filed as ChuckBuilds/ledmatrix-plugins#464 and #465.

* fix(harness): give an empty-looking mode a few frames before warning about it

check_plugin's "drew nothing but display() returned X" warning fired on a single
frame, rendered with force_clear=True, under a frozen clock. All three defeat a
scrolling plugin, whose first frame is legitimately its blank scroll-in buffer.
Across 44 first-party plugins, 60 of 76 warnings were false -- the rate at which
people stop reading a warning, which matters because the true positives are
real: a mode that draws nothing and does not return False holds a blank panel
for its whole display duration.

An apparently-empty frame is now re-driven for up to 48 more frames with
force_clear=False (force_clear means "reset the scroll", so repeating it would
redraw frame 1 for ever) and with the clock advancing -- freezegun's factory
where time is frozen, a real sleep where it is not, since scroll position is
usually a function of elapsed time. The first frame that draws content replaces
the result.

The clock is moved back afterwards. It is shared by every render in the matrix,
so time borrowed by the probe leaked into later modes and drifted their goldens
-- f1_upcoming picked up 5 spurious drifts before this was restored.

Measured on the rig:

                        empty warns          check
                        before  after
  f1-scoreboard            42      0    48 PASS / 0 FAIL, goldens intact
  ledmatrix-elections      16      0    16 PASS / 0 FAIL
  on-air                    8      8    true positive, kept
  nfl-draft                 8      8    true positive, kept
  clock-simple/geochron/    0      0    unchanged
  christmas-countdown

58 false positives gone, both true positives kept, no golden regressions. Cost
is confined to modes that really are blank: plugins that draw immediately are
unchanged (clock-simple and tide-display still 2s), while on-air -- eight
deliberately blank modes -- goes to 21s.

Closes #527.

* fix(harness): load nested schema defaults, and merge caller config at leaf level

load_config_defaults read only top-level properties. An object property carries
its defaults on its children, not on itself, so everything nested was dropped --
2,386 defaults across 37 of 44 plugins, soccer-scoreboard alone losing 539 of
565. render_plugin_matrix's comment says the plugin then "behaves like a real
install", which for most of the fleet it did not.

_defaults_from_properties now recurses. merge_config deep-merges the caller's
config onto the result so an override lands at the leaf: a shallow merge would
let -c '{"nhl": {"enabled": true}}' replace the whole nhl subtree and discard
every other nhl default, which is the same class of bug being fixed here.

Measured before/after across all 49 installed plugins on the rig: **no render
changed** -- identical PASS/FAIL counts, byte-identical output, goldens intact.
Plugins already fall back to the same values internally via config.get(key,
default), so supplying them explicitly agrees with what they were doing. The
defaults really are arriving now:

  ufc-scoreboard        9 -> 87 defaults
  ledmatrix-flights    51 -> 95
  masters-tournament   10 -> 51
  cricket-scoreboard   22 -> 50
  tide-display         12 -> 18

and hockey-scoreboard, which used to load nhl.enabled=None, now gets
nhl.enabled=True with its full display_modes block.

Caveat worth carrying: the eight plugins with the most nested config
(soccer, baseball, basketball, hockey, lacrosse, football, afl, nrl -- 1,634 of
the 2,386 dropped defaults, 68%) could not be measured. They import
src.common.sports_shared, which the test rig's core branch predates, so they
fail to load there identically before and after. Re-run this comparison against
a core that has that module before trusting the "nothing changed" result for
them; those are exactly the plugins whose renders should change most.

Closes #531.

* refactor: narrow the exception handlers this branch introduced

Codacy flagged the new code; it passes on other recent PRs, so the finding is
mine. Four of the five broad `except Exception` clauses I added were catching
far more than they needed to, which is the same shape as several bugs this
branch fixes -- hello-world's TypeError sat invisible for exactly this reason.

  freezer() / move_to() / tick()   -> (AttributeError, TypeError, ValueError)
  cache_manager.delete()           -> (OSError, AttributeError, KeyError)

The fifth stays broad and now says why: it wraps a call into a plugin's own
display(), which can raise anything, and the first frame has already rendered --
so a failure there must not turn a good result into an error.

Verified against a checkout of main: f1-scoreboard 48 PASS / 0 FAIL with 0 empty
warnings, on-air keeps its 8 true positives, clock-simple 8 PASS. geochron shows
7 golden drifts both before and after this branch, so it is not from these
changes -- its committed goldens predate #521's 1-bit text rendering.

* fix: resolve CodeRabbit review and Codacy findings on #534

CodeRabbit raised six; all six were real.

The test double had drifted ahead of production. VisualTestDisplayManager
accepted set_scrolling_state(frame_hold=...) while DisplayManager did not,
so such a call passed every harness run and would raise TypeError on the
panel -- the one failure a safety harness exists to prevent. frame_hold
belongs to the change that adds it to DisplayManager (#523), so it moves
there and the double matches main again.

The harness swallowed exceptions from re-rendered frames. _settle_loop
re-renders a mode that came back blank, to give a scroll time to draw;
returning silently on a crash meant a mode that renders one good frame
and then explodes was reported as passing. Recorded on result.error now,
keeping the captured frame so the failure stays inspectable.

starlark-apps display() returned True after _display_frame() failed, so
the controller held a dead frame for the whole display_duration instead
of rotating on. _display_frame now returns bool on all three paths.

run_plugin_tests.py used env.setdefault for PYTHONPATH and
LEDMATRIX_CORE, so an inherited value won and the subprocess imported a
different core than the one under test -- ledmatrix-plugins#467 exactly.
Prepends PROJECT_ROOT and sets LEDMATRIX_CORE unconditionally.

The on-demand mailbox is polled after every frame, ~125x/second on a
scrolling mode, and the read is deliberately uncached, so it was that
many disk reads per second to find nothing. Floored at 250ms, which is
imperceptible for a web-UI click. Consuming it also deleted whatever was
present rather than what had just been processed, so a request posted
while the previous one was in flight was thrown away and never ran; the
delete is now keyed by request_id. That narrows the window rather than
closing it -- a true atomic claim needs a primitive the cache layer does
not offer, and the code says so rather than implying otherwise.

Codacy's 2 criticals were bandit B404/B603 on the subprocess call added
to run_plugin_tests.py. Fixed interpreter, argument list, no shell;
annotated with the repo's existing nosec convention. Bandit is clean on
the file.

Adds test/test_on_demand_mailbox.py (8), test_starlark_display_contract.py
(4) and two settle cases in test_harness_empty_claimed.py. 4, 4 and 2 of
those fail against the pre-fix code. Full suite: 3961 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* chore: satisfy Codacy's subprocess checks on the new test runner

Codacy runs Bandit and Opengrep (its Semgrep fork). The new
subprocess.run in scripts/run_plugin_tests.py trips three patterns, on
two different lines:

  Bandit   B404 on the import, B603 on the call
  Opengrep dangerous-subprocess-use-audit          on the run( line
           dangerous-subprocess-use-tainted-env-args on the argv line

A nosemgrep applies only to its own line, so the call line and the argv
line each need one; a single comment on the call covered neither rule
fully. Suppression is the right answer here rather than a rewrite: the
interpreter is sys.executable, the arguments are a list, and no shell is
involved, so there is nothing to word-split or expand.

Matches the pair the rest of the repo already uses for this shape --
permission_utils.py, plugin_loader.py, install_dependencies_apt.py.

Codacy: 0 new issues, up to standards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* chore: leave visual_display_manager untouched so #523 can merge

The only change this branch made to that file was a docstring, and it
collided with #523's rewrite of the same method -- so #534 and #523 each
merged cleanly against main but conflicted with each other. Reverted to
main's text; #523 owns this method and adds frame_hold to it.

The note the docstring carried ('frame_hold arrives in #523') would have
been stale the moment #523 landed anyway. The parity test in #523 is
what actually keeps the two signatures honest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* chore: add the Ruff suppression nosec/nosemgrep do not cover

Ruff reports S603 on the same call Bandit and Opengrep do, and none of
the three suppressions covers the others. Confirmed the precondition
first: path comes from discover_plugin_tests(), which globs test files
inside the repo, and the call is a fixed interpreter with a list argv
and no shell.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-07 13:37:38 -04:00

1074 lines
41 KiB
Python

"""
Starlark Apps Plugin for LEDMatrix
Manages and displays Starlark (.star) apps from Tronbyte/Tidbyt community.
Provides seamless widget import without modification.
API Version: 1.0.0
"""
import json
import os
import re
import time
import fcntl
from pathlib import Path
from typing import Dict, Any, Optional, List, Tuple
from PIL import Image
from src.plugin_system.base_plugin import BasePlugin, VegasDisplayMode
from src.logging_config import get_logger
from pixlet_renderer import PixletRenderer
from frame_extractor import FrameExtractor
logger = get_logger(__name__)
class StarlarkApp:
"""Represents a single installed Starlark app."""
def __init__(self, app_id: str, app_dir: Path, manifest: Dict[str, Any]):
"""
Initialize a Starlark app instance.
Args:
app_id: Unique identifier for this app
app_dir: Directory containing the app files
manifest: App metadata from manifest
"""
self.app_id = app_id
self.app_dir = app_dir
self.manifest = manifest
self.star_file = app_dir / manifest.get("star_file", f"{app_id}.star")
self.config_file = app_dir / "config.json"
self.schema_file = app_dir / "schema.json"
self.cache_file = app_dir / "cached_render.webp"
# Load app configuration and schema
self.config = self._load_config()
self.schema = self._load_schema()
# Merge schema defaults into config for any missing fields
self._merge_schema_defaults()
# Runtime state
self.frames: Optional[List[Tuple[Image.Image, int]]] = None
self.current_frame_index = 0
self.last_frame_time = 0
self.last_render_time = 0
def _load_config(self) -> Dict[str, Any]:
"""Load app configuration from config.json."""
if self.config_file.exists():
try:
with open(self.config_file, 'r') as f:
return json.load(f)
except (OSError, json.JSONDecodeError) as e:
logger.warning(f"Could not load config for {self.app_id}: {e}")
return {}
def _load_schema(self) -> Optional[Dict[str, Any]]:
"""Load app schema from schema.json."""
if self.schema_file.exists():
try:
with open(self.schema_file, 'r') as f:
return json.load(f)
except (OSError, json.JSONDecodeError) as e:
logger.warning(f"Could not load schema for {self.app_id}: {e}")
return None
def _merge_schema_defaults(self) -> None:
"""
Merge schema default values into config for any missing fields.
This ensures existing apps get defaults when schemas are updated with new fields.
"""
if not self.schema:
return
# Get fields from schema (handles both 'fields' and 'schema' keys)
fields = self.schema.get('fields') or self.schema.get('schema') or []
defaults_added = False
for field in fields:
if isinstance(field, dict) and 'id' in field and 'default' in field:
field_id = field['id']
# Only add if not already present in config
if field_id not in self.config:
self.config[field_id] = field['default']
defaults_added = True
logger.debug(f"Added default value for {self.app_id}.{field_id}: {field['default']}")
# Save config if we added any defaults
if defaults_added:
self.save_config()
def _validate_config(self) -> Optional[str]:
"""
Validate config values to prevent injection and ensure data integrity.
Returns:
Error message if validation fails, None if valid
"""
for key, value in self.config.items():
# Validate key format
if not re.match(r'^[a-zA-Z_][a-zA-Z0-9_]{0,63}$', key):
return f"Invalid config key format: {key}"
# Validate location fields (JSON format)
if isinstance(value, str) and value.strip().startswith('{'):
try:
loc = json.loads(value)
except json.JSONDecodeError as e:
return f"Invalid JSON for key {key}: {e}"
# Validate lat/lng if present
try:
if 'lat' in loc:
lat = float(loc['lat'])
if not -90 <= lat <= 90:
return f"Latitude {lat} out of range [-90, 90] for key {key}"
if 'lng' in loc:
lng = float(loc['lng'])
if not -180 <= lng <= 180:
return f"Longitude {lng} out of range [-180, 180] for key {key}"
except ValueError as e:
return f"Invalid numeric value for {key}: {e}"
return None
def save_config(self) -> bool:
"""Save current configuration to file with validation."""
try:
# Validate config before saving
error = self._validate_config()
if error:
logger.error(f"Config validation failed for {self.app_id}: {error}")
return False
with open(self.config_file, 'w') as f:
json.dump(self.config, f, indent=2)
return True
except Exception as e:
logger.exception(f"Could not save config for {self.app_id}: {e}")
return False
def is_enabled(self) -> bool:
"""Check if app is enabled."""
return self.manifest.get("enabled", True)
def get_render_interval(self) -> int:
"""Get render interval in seconds."""
default = 300
try:
value = self.manifest.get("render_interval", default)
interval = int(value)
except (ValueError, TypeError):
interval = default
# Clamp to safe range: min 5, max 3600
return max(5, min(interval, 3600))
def get_display_duration(self) -> int:
"""Get display duration in seconds."""
default = 15
try:
value = self.manifest.get("display_duration", default)
duration = int(value)
except (ValueError, TypeError):
duration = default
# Clamp to safe range: min 1, max 600
return max(1, min(duration, 600))
def should_render(self, current_time: float) -> bool:
"""Check if app should be re-rendered based on interval."""
interval = self.get_render_interval()
return (current_time - self.last_render_time) >= interval
class StarlarkAppsPlugin(BasePlugin):
"""
Starlark Apps Manager plugin.
Manages Starlark (.star) apps and renders them using Pixlet.
Each installed app becomes a dynamic display mode.
"""
def __init__(self, plugin_id: str, config: Dict[str, Any],
display_manager, cache_manager, plugin_manager):
"""Initialize the Starlark Apps plugin."""
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
# Initialize components
self.pixlet = PixletRenderer(
pixlet_path=config.get("pixlet_path"),
timeout=config.get("render_timeout", 30)
)
self.extractor = FrameExtractor(
default_frame_delay=config.get("default_frame_delay", 50)
)
# App storage
self.apps_dir = self._get_apps_directory()
self.manifest_file = self.apps_dir / "manifest.json"
self.apps: Dict[str, StarlarkApp] = {}
# Display state
self.current_app: Optional[StarlarkApp] = None
self.last_update_check = 0
# Check Pixlet availability
if not self.pixlet.is_available():
self.logger.error("Pixlet not available - Starlark apps will not work")
self.logger.error("Install Pixlet or place bundled binary in bin/pixlet/")
else:
version = self.pixlet.get_version()
self.logger.info(f"Pixlet available: {version}")
# Calculate optimal magnification based on display size
self.calculated_magnify = self._calculate_optimal_magnify()
if self.calculated_magnify > 1:
self.logger.info(f"Display size: {self.display_manager.matrix.width}x{self.display_manager.matrix.height}, "
f"recommended magnify: {self.calculated_magnify}")
# Load installed apps
self._load_installed_apps()
self.logger.info(f"Starlark Apps plugin initialized with {len(self.apps)} apps")
@property
def modes(self) -> List[str]:
"""
Return list of display modes (one per installed Starlark app).
This allows each installed app to appear as a separate display mode
in the schedule/rotation system.
Returns:
List of app IDs that can be used as display modes
"""
# Return list of enabled app IDs as display modes
return [app.app_id for app in self.apps.values() if app.is_enabled()]
def validate_config(self) -> bool:
"""
Validate plugin configuration.
Ensures required configuration values are valid for Starlark apps.
Returns:
True if configuration is valid, False otherwise
"""
# Call parent validation first
if not super().validate_config():
return False
# Validate magnify range (0-8)
if "magnify" in self.config:
magnify = self.config["magnify"]
if not isinstance(magnify, int) or magnify < 0 or magnify > 8:
self.logger.error("magnify must be an integer between 0 and 8")
return False
# Validate render_timeout
if "render_timeout" in self.config:
timeout = self.config["render_timeout"]
if not isinstance(timeout, (int, float)) or timeout < 5 or timeout > 120:
self.logger.error("render_timeout must be a number between 5 and 120")
return False
# Validate cache_ttl
if "cache_ttl" in self.config:
ttl = self.config["cache_ttl"]
if not isinstance(ttl, (int, float)) or ttl < 60 or ttl > 3600:
self.logger.error("cache_ttl must be a number between 60 and 3600")
return False
# Validate scale_method
if "scale_method" in self.config:
method = self.config["scale_method"]
valid_methods = ["nearest", "bilinear", "bicubic", "lanczos"]
if method not in valid_methods:
self.logger.error(f"scale_method must be one of: {', '.join(valid_methods)}")
return False
# Validate default_frame_delay
if "default_frame_delay" in self.config:
delay = self.config["default_frame_delay"]
if not isinstance(delay, (int, float)) or delay < 16 or delay > 1000:
self.logger.error("default_frame_delay must be a number between 16 and 1000")
return False
return True
def _calculate_optimal_magnify(self) -> int:
"""
Calculate optimal magnification factor based on display dimensions.
Tronbyte apps are designed for 64x32 displays.
This calculates what magnification would best fit the current display.
Returns:
Recommended magnify value (1-8)
"""
try:
display_width = self.display_manager.matrix.width
display_height = self.display_manager.matrix.height
# Tronbyte native resolution
NATIVE_WIDTH = 64
NATIVE_HEIGHT = 32
# Calculate scale factors for width and height
width_scale = display_width / NATIVE_WIDTH
height_scale = display_height / NATIVE_HEIGHT
# Use the smaller scale to ensure content fits
# (prevents overflow on one dimension)
scale_factor = min(width_scale, height_scale)
# Round down to get integer magnify value
magnify = int(scale_factor)
# Clamp to reasonable range (1-8)
magnify = max(1, min(8, magnify))
self.logger.debug(f"Display: {display_width}x{display_height}, "
f"Native: {NATIVE_WIDTH}x{NATIVE_HEIGHT}, "
f"Calculated magnify: {magnify}")
return magnify
except Exception as e:
self.logger.warning(f"Could not calculate magnify: {e}")
return 1
def get_magnify_recommendation(self) -> Dict[str, Any]:
"""
Get detailed magnification recommendation for current display.
Returns:
Dictionary with recommendation details
"""
try:
display_width = self.display_manager.matrix.width
display_height = self.display_manager.matrix.height
NATIVE_WIDTH = 64
NATIVE_HEIGHT = 32
width_scale = display_width / NATIVE_WIDTH
height_scale = display_height / NATIVE_HEIGHT
# Calculate for different magnify values
recommendations = []
for magnify in range(1, 9):
render_width = NATIVE_WIDTH * magnify
render_height = NATIVE_HEIGHT * magnify
# Check if this magnify fits perfectly
perfect_fit = (render_width == display_width and render_height == display_height)
# Check if scaling is needed
needs_scaling = (render_width != display_width or render_height != display_height)
# Calculate quality score (1-100)
if perfect_fit:
quality_score = 100
elif not needs_scaling:
quality_score = 95
else:
# Score based on how close to display size
width_ratio = min(render_width, display_width) / max(render_width, display_width)
height_ratio = min(render_height, display_height) / max(render_height, display_height)
quality_score = int((width_ratio + height_ratio) / 2 * 100)
recommendations.append({
'magnify': magnify,
'render_size': f"{render_width}x{render_height}",
'perfect_fit': perfect_fit,
'needs_scaling': needs_scaling,
'quality_score': quality_score,
'recommended': magnify == self.calculated_magnify
})
return {
'display_size': f"{display_width}x{display_height}",
'native_size': f"{NATIVE_WIDTH}x{NATIVE_HEIGHT}",
'calculated_magnify': self.calculated_magnify,
'width_scale': round(width_scale, 2),
'height_scale': round(height_scale, 2),
'recommendations': recommendations
}
except Exception as e:
self.logger.exception(f"Error getting magnify recommendation: {e}")
return {
'display_size': 'unknown',
'calculated_magnify': 1,
'recommendations': []
}
def _get_effective_magnify(self) -> int:
"""
Get the effective magnify value to use for rendering.
Priority:
1. User-configured magnify (if valid and in range 1-8)
2. Auto-calculated magnify
Returns:
Magnify value to use
"""
config_magnify = self.config.get("magnify")
# Validate and clamp config_magnify
if config_magnify is not None:
try:
# Convert to int if possible
config_magnify = int(config_magnify)
# Clamp to safe range (1-8)
if 1 <= config_magnify <= 8:
return config_magnify
except (ValueError, TypeError):
# Non-numeric value, fall through to calculated
pass
# Fall back to auto-calculated value
return self.calculated_magnify
def _get_apps_directory(self) -> Path:
"""Get the directory for storing Starlark apps."""
try:
# Try to find project root
current_dir = Path(__file__).resolve().parent
project_root = current_dir.parent.parent
apps_dir = project_root / "starlark-apps"
except Exception:
# Fallback to current working directory
apps_dir = Path.cwd() / "starlark-apps"
# Create directory if it doesn't exist
apps_dir.mkdir(parents=True, exist_ok=True)
return apps_dir
def _sanitize_app_id(self, app_id: str) -> str:
"""
Sanitize app_id into a safe slug for use in file paths.
Args:
app_id: Original app identifier
Returns:
Sanitized slug containing only [a-z0-9_.-] characters
"""
if not app_id:
raise ValueError("app_id cannot be empty")
# Replace invalid characters with underscore
# Allow only: lowercase letters, digits, underscore, period, hyphen
safe_slug = re.sub(r'[^a-z0-9_.-]', '_', app_id.lower())
# Remove leading/trailing dots, underscores, or hyphens
safe_slug = safe_slug.strip('._-')
# Ensure it's not empty after sanitization
if not safe_slug:
raise ValueError(f"app_id '{app_id}' becomes empty after sanitization")
return safe_slug
def _verify_path_safety(self, path: Path, base_dir: Path) -> None:
"""
Verify that a path is within the base directory to prevent path traversal.
Args:
path: Path to verify
base_dir: Base directory that path must be within
Raises:
ValueError: If path escapes the base directory
"""
try:
resolved_path = path.resolve()
resolved_base = base_dir.resolve()
# Check if path is relative to base directory
if not resolved_path.is_relative_to(resolved_base):
raise ValueError(
f"Path traversal detected: {resolved_path} is not within {resolved_base}"
)
except (ValueError, AttributeError) as e:
# AttributeError for Python < 3.9 where is_relative_to doesn't exist
# Fallback: check if resolved path starts with resolved base
resolved_path = path.resolve()
resolved_base = base_dir.resolve()
try:
resolved_path.relative_to(resolved_base)
except ValueError:
raise ValueError(
f"Path traversal detected: {resolved_path} is not within {resolved_base}"
) from e
def _load_installed_apps(self) -> None:
"""Load all installed apps from manifest."""
if not self.manifest_file.exists():
# Create initial manifest
self._save_manifest({"apps": {}})
return
try:
with open(self.manifest_file, 'r') as f:
manifest = json.load(f)
apps_data = manifest.get("apps", {})
for app_id, app_manifest in apps_data.items():
try:
# Sanitize app_id to prevent path traversal
safe_app_id = self._sanitize_app_id(app_id)
app_dir = (self.apps_dir / safe_app_id).resolve()
# Verify path safety
self._verify_path_safety(app_dir, self.apps_dir)
except ValueError as e:
self.logger.warning(f"Invalid app_id '{app_id}': {e}")
continue
if not app_dir.exists():
self.logger.warning(f"App directory missing: {app_id}")
continue
try:
# Use safe_app_id for internal storage to match directory structure
app = StarlarkApp(safe_app_id, app_dir, app_manifest)
self.apps[safe_app_id] = app
self.logger.debug(f"Loaded app: {app_id} (sanitized: {safe_app_id})")
except Exception as e:
self.logger.exception(f"Error loading app {app_id}: {e}")
self.logger.info(f"Loaded {len(self.apps)} Starlark apps")
except Exception as e:
self.logger.exception(f"Error loading apps manifest: {e}")
def _save_manifest(self, manifest: Dict[str, Any]) -> bool:
"""
Save apps manifest to file with file locking to prevent race conditions.
Acquires exclusive lock on manifest file before writing to prevent concurrent modifications.
"""
temp_file = None
lock_fd = None
try:
# Create parent directory if needed
self.manifest_file.parent.mkdir(parents=True, exist_ok=True)
# Open manifest file for locking (create if doesn't exist, don't truncate)
# Use os.open with O_CREAT | O_RDWR to create if missing, but don't truncate
lock_fd = os.open(str(self.manifest_file), os.O_CREAT | os.O_RDWR, 0o644)
# Acquire exclusive lock on manifest file BEFORE creating temp file
# This serializes all writers and prevents concurrent races
fcntl.flock(lock_fd, fcntl.LOCK_EX)
try:
# Now that we hold the lock, create and write temp file
temp_file = self.manifest_file.with_suffix('.tmp')
with open(temp_file, 'w') as f:
json.dump(manifest, f, indent=2)
f.flush()
os.fsync(f.fileno()) # Ensure data is written to disk
# Atomic rename (overwrites destination) while still holding lock
temp_file.replace(self.manifest_file)
return True
finally:
# Release lock
fcntl.flock(lock_fd, fcntl.LOCK_UN)
os.close(lock_fd)
except (OSError, IOError, json.JSONDecodeError, ValueError) as e:
self.logger.exception("Error saving manifest while writing manifest file", exc_info=True)
# Clean up temp file if it exists
if temp_file is not None and temp_file.exists():
try:
temp_file.unlink()
except Exception as cleanup_exc:
self.logger.warning(f"Failed to clean up temp file {temp_file}: {cleanup_exc}")
# Clean up lock fd if still open
if lock_fd is not None:
try:
os.close(lock_fd)
except Exception as cleanup_exc:
self.logger.warning(f"Failed to close lock file descriptor: {cleanup_exc}")
return False
def _update_manifest_safe(self, updater_fn) -> bool:
"""
Safely update manifest with file locking to prevent race conditions.
Holds exclusive lock for entire read-modify-write cycle.
Args:
updater_fn: Function that takes manifest dict and modifies it in-place
Returns:
True if successful, False otherwise
"""
lock_fd = None
temp_file = None
try:
# Create parent directory if needed
self.manifest_file.parent.mkdir(parents=True, exist_ok=True)
# Open manifest file for locking (create if doesn't exist, don't truncate)
lock_fd = os.open(str(self.manifest_file), os.O_CREAT | os.O_RDWR, 0o644)
# Acquire exclusive lock for entire read-modify-write cycle
fcntl.flock(lock_fd, fcntl.LOCK_EX)
try:
# Read current manifest while holding exclusive lock
if self.manifest_file.exists() and self.manifest_file.stat().st_size > 0:
with open(self.manifest_file, 'r') as f:
manifest = json.load(f)
else:
# Empty or non-existent file, start with default structure
manifest = {"apps": {}}
# Apply updates while still holding lock
updater_fn(manifest)
# Write back to temp file, then atomic replace (still holding lock)
temp_file = self.manifest_file.with_suffix('.tmp')
with open(temp_file, 'w') as f:
json.dump(manifest, f, indent=2)
f.flush()
os.fsync(f.fileno())
# Atomic rename while still holding lock
temp_file.replace(self.manifest_file)
return True
finally:
# Release lock
fcntl.flock(lock_fd, fcntl.LOCK_UN)
os.close(lock_fd)
except (OSError, IOError, json.JSONDecodeError, ValueError) as e:
self.logger.exception("Error updating manifest during read-modify-write cycle", exc_info=True)
# Clean up temp file if it exists
if temp_file is not None and temp_file.exists():
try:
temp_file.unlink()
except Exception as cleanup_exc:
self.logger.warning(f"Failed to clean up temp file {temp_file}: {cleanup_exc}")
# Clean up lock fd if still open
if lock_fd is not None:
try:
os.close(lock_fd)
except Exception as cleanup_exc:
self.logger.warning(f"Failed to close lock file descriptor: {cleanup_exc}")
return False
def update(self) -> None:
"""Update method - check if apps need re-rendering."""
current_time = time.time()
# Check apps that need re-rendering based on their intervals
if self.config.get("auto_refresh_apps", True):
for app in self.apps.values():
if app.is_enabled() and app.should_render(current_time):
self._render_app(app, force=False)
def display(self, force_clear: bool = False) -> bool:
"""
Display current Starlark app.
This method is called during the display rotation.
Displays frames from the currently active app.
Returns False when there is no app to show -- which is the state of
every install without Pixlet, and of a fresh one before any app is
added. The display controller only skips a mode on a boolean False
(it checks isinstance(result, bool)), so returning None held a black
panel for the full display_duration instead of rotating on.
"""
try:
if force_clear:
self.display_manager.clear()
# If no current app, try to select one
if not self.current_app:
self._select_next_app()
if not self.current_app:
# No apps available
self.logger.debug("No Starlark apps to display")
return False
# Render app if needed
if not self.current_app.frames:
success = self._render_app(self.current_app, force=True)
if not success:
self.logger.error(f"Failed to render app: {self.current_app.app_id}")
return False
# Display current frame. The result is propagated: a failed frame
# update is not a displayed frame, and returning True regardless
# told the controller the mode had rendered, so it held the dead
# frame for the whole display_duration instead of rotating on.
return self._display_frame()
except Exception as e:
self.logger.error(f"Error displaying Starlark app: {e}")
return False
def _select_next_app(self) -> None:
"""Select the next enabled app for display."""
enabled_apps = [app for app in self.apps.values() if app.is_enabled()]
if not enabled_apps:
self.current_app = None
return
# Simple rotation - could be enhanced with priorities
if self.current_app and self.current_app in enabled_apps:
current_idx = enabled_apps.index(self.current_app)
next_idx = (current_idx + 1) % len(enabled_apps)
self.current_app = enabled_apps[next_idx]
else:
self.current_app = enabled_apps[0]
self.logger.debug(f"Selected app for display: {self.current_app.app_id}")
def _render_app(self, app: StarlarkApp, force: bool = False) -> bool:
"""
Render a Starlark app using Pixlet.
Args:
app: App to render
force: Force render even if cached
Returns:
True if successful
"""
try:
current_time = time.time()
# Check cache
use_cache = self.config.get("cache_rendered_output", True)
cache_ttl = self.config.get("cache_ttl", 300)
if (not force and use_cache and app.cache_file.exists() and
(current_time - app.last_render_time) < cache_ttl):
# Use cached render
self.logger.debug(f"Using cached render for: {app.app_id}")
return self._load_frames_from_cache(app)
# Render with Pixlet
self.logger.info(f"Rendering app: {app.app_id}")
# Get effective magnification factor (config or auto-calculated)
magnify = self._get_effective_magnify()
self.logger.debug(f"Using magnify={magnify} for {app.app_id}")
# Filter out LEDMatrix-internal timing keys before passing to pixlet
INTERNAL_KEYS = {'render_interval', 'display_duration'}
pixlet_config = {k: v for k, v in app.config.items() if k not in INTERNAL_KEYS}
success, error = self.pixlet.render(
star_file=str(app.star_file),
output_path=str(app.cache_file),
config=pixlet_config,
magnify=magnify
)
if not success:
self.logger.error(f"Pixlet render failed: {error}")
return False
# Extract frames
success = self._load_frames_from_cache(app)
if success:
app.last_render_time = current_time
return success
except Exception as e:
self.logger.error(f"Error rendering app {app.app_id}: {e}")
return False
def _load_frames_from_cache(self, app: StarlarkApp) -> bool:
"""Load frames from cached WebP file."""
try:
success, frames, error = self.extractor.load_webp(str(app.cache_file))
if not success:
self.logger.error(f"Frame extraction failed: {error}")
return False
# Scale frames if needed
if self.config.get("scale_output", True):
width = self.display_manager.matrix.width
height = self.display_manager.matrix.height
# Get scaling method from config
scale_method_str = self.config.get("scale_method", "nearest")
scale_method_map = {
"nearest": Image.Resampling.NEAREST,
"bilinear": Image.Resampling.BILINEAR,
"bicubic": Image.Resampling.BICUBIC,
"lanczos": Image.Resampling.LANCZOS
}
scale_method = scale_method_map.get(scale_method_str, Image.Resampling.NEAREST)
# Check if we should center instead of scale
if self.config.get("center_small_output", False):
frames = self.extractor.center_frames(frames, width, height)
else:
frames = self.extractor.scale_frames(frames, width, height, scale_method)
# Optimize frames to limit memory usage (max_frames=None means no limit)
max_frames = self.config.get("max_frames")
if max_frames is not None:
frames = self.extractor.optimize_frames(frames, max_frames=max_frames)
app.frames = frames
app.current_frame_index = 0
app.last_frame_time = time.time()
self.logger.debug(f"Loaded {len(frames)} frames for {app.app_id}")
return True
except Exception as e:
self.logger.error(f"Error loading frames for {app.app_id}: {e}")
return False
def _display_frame(self) -> bool:
"""Display the current frame of the current app.
:returns: whether a frame actually reached the display manager.
"""
if not self.current_app or not self.current_app.frames:
return False
try:
current_time = time.time()
frame, delay_ms = self.current_app.frames[self.current_app.current_frame_index]
# Set frame on display manager
self.display_manager.image = frame
self.display_manager.update_display()
# Check if it's time to advance to next frame
delay_seconds = delay_ms / 1000.0
if (current_time - self.current_app.last_frame_time) >= delay_seconds:
self.current_app.current_frame_index = (
(self.current_app.current_frame_index + 1) % len(self.current_app.frames)
)
self.current_app.last_frame_time = current_time
return True
except Exception as e:
self.logger.error(f"Error displaying frame: {e}")
return False
def install_app(self, app_id: str, star_file_path: str, metadata: Optional[Dict[str, Any]] = None, assets_dir: Optional[str] = None) -> bool:
"""
Install a new Starlark app.
Args:
app_id: Unique identifier for the app
star_file_path: Path to .star file to install
metadata: Optional metadata (name, description, etc.)
assets_dir: Optional directory containing assets (images/, sources/, etc.)
Returns:
True if successful
"""
try:
import shutil
# Sanitize app_id to prevent path traversal
safe_app_id = self._sanitize_app_id(app_id)
# Create app directory with resolved path
app_dir = (self.apps_dir / safe_app_id).resolve()
# Verify path safety BEFORE creating directories
self._verify_path_safety(app_dir, self.apps_dir)
app_dir.mkdir(parents=True, exist_ok=True)
# Copy .star file with sanitized app_id
star_dest = app_dir / f"{safe_app_id}.star"
# Verify star_dest path safety
self._verify_path_safety(star_dest, self.apps_dir)
shutil.copy2(star_file_path, star_dest)
# Copy asset directories if provided (images/, sources/, etc.)
if assets_dir and Path(assets_dir).exists():
assets_path = Path(assets_dir)
for item in assets_path.iterdir():
if item.is_dir():
# Copy entire directory (e.g., images/, sources/)
dest_dir = app_dir / item.name
# Verify dest_dir path safety
self._verify_path_safety(dest_dir, self.apps_dir)
if dest_dir.exists():
shutil.rmtree(dest_dir)
shutil.copytree(item, dest_dir)
self.logger.debug(f"Copied assets directory: {item.name}")
self.logger.info(f"Installed assets for {app_id}")
# Create app manifest entry
app_manifest = {
"name": metadata.get("name", app_id) if metadata else app_id,
"original_id": app_id, # Store original for reference
"star_file": f"{safe_app_id}.star",
"enabled": True,
"render_interval": metadata.get("render_interval", 300) if metadata else 300,
"display_duration": metadata.get("display_duration", 15) if metadata else 15
}
# Try to extract schema
_, schema, _ = self.pixlet.extract_schema(str(star_dest))
if schema:
schema_path = app_dir / "schema.json"
# Verify schema path safety
self._verify_path_safety(schema_path, self.apps_dir)
with open(schema_path, 'w') as f:
json.dump(schema, f, indent=2)
# Create default config — pre-populate with schema defaults
default_config = {}
if schema:
fields = schema.get('fields') or schema.get('schema') or []
for field in fields:
if isinstance(field, dict) and 'id' in field and 'default' in field:
default_config[field['id']] = field['default']
config_path = app_dir / "config.json"
# Verify config path safety
self._verify_path_safety(config_path, self.apps_dir)
with open(config_path, 'w') as f:
json.dump(default_config, f, indent=2)
# Update manifest (use safe_app_id as key to match directory)
def update_fn(manifest):
manifest["apps"][safe_app_id] = app_manifest
if not self._update_manifest_safe(update_fn):
self.logger.error(f"Failed to update manifest for {app_id}")
return False
# Create app instance (use safe_app_id for internal key, original for display)
app = StarlarkApp(safe_app_id, app_dir, app_manifest)
self.apps[safe_app_id] = app
self.logger.info(f"Installed Starlark app: {app_id} (sanitized: {safe_app_id})")
return True
except Exception as e:
self.logger.error(f"Error installing app {app_id}: {e}")
return False
def uninstall_app(self, app_id: str) -> bool:
"""
Uninstall a Starlark app.
Args:
app_id: App to uninstall
Returns:
True if successful
"""
try:
import shutil
if app_id not in self.apps:
self.logger.warning(f"App not found: {app_id}")
return False
# Remove from current app if selected
if self.current_app and self.current_app.app_id == app_id:
self.current_app = None
# Get app reference before removing from dict
app = self.apps.get(app_id)
# Update manifest FIRST (before modifying filesystem)
def update_fn(manifest):
if app_id in manifest["apps"]:
del manifest["apps"][app_id]
if not self._update_manifest_safe(update_fn):
self.logger.error(f"Failed to update manifest when uninstalling {app_id}")
return False
# Remove from apps dict
self.apps.pop(app_id)
# Remove directory (after manifest update succeeds)
if app and app.app_dir.exists():
shutil.rmtree(app.app_dir)
self.logger.info(f"Uninstalled Starlark app: {app_id}")
return True
except Exception as e:
self.logger.error(f"Error uninstalling app {app_id}: {e}")
return False
def get_display_duration(self) -> float:
"""Get display duration for current app."""
if self.current_app:
return float(self.current_app.get_display_duration())
return self.config.get('display_duration', 15.0)
# ─── Vegas Mode Integration ──────────────────────────────────────
def get_vegas_content(self) -> Optional[List[Image.Image]]:
"""Return rendered frames from enabled starlark apps for vegas scroll."""
images = []
for app in self.apps.values():
if not app.is_enabled():
continue
# Use cached frames if available
if app.frames:
images.extend([frame for frame, delay in app.frames])
else:
# Try to render and extract frames
if self._render_app(app):
if app.frames:
images.extend([frame for frame, delay in app.frames])
return images if images else None
def get_vegas_content_type(self) -> str:
"""Indicate the type of content for Vegas scroll."""
return "multi"
def get_vegas_display_mode(self) -> VegasDisplayMode:
"""Get the display mode for Vegas scroll integration."""
return VegasDisplayMode.FIXED_SEGMENT
def get_info(self) -> Dict[str, Any]:
"""Return plugin info for web UI."""
info = super().get_info()
info.update({
'pixlet_available': self.pixlet.is_available(),
'pixlet_version': self.pixlet.get_version(),
'installed_apps': len(self.apps),
'enabled_apps': len([a for a in self.apps.values() if a.is_enabled()]),
'current_app': self.current_app.app_id if self.current_app else None,
'apps': {
app_id: {
'name': app.manifest.get('name', app_id),
'enabled': app.is_enabled(),
'has_frames': app.frames is not None
}
for app_id, app in self.apps.items()
}
})
return info