Compare commits

...
3 Commits
Author SHA1 Message Date
ChuckandClaude Opus 5 793b988d33 fix(starlark): toggle the app id the list published, and store a relocatable star_file (#537)
The two review nitpicks left over from #535. Both are still on main after
that merge; the five findings alongside them landed with it.

**The toggle could not find what the list had just shown.**
`_starlark_virtual_plugins` publishes the raw manifest key as
`starlark:<key>`, and `_toggle_starlark_app` passed it back through
`_validate_and_sanitize_app_id`, which lowercases and rewrites every
character outside `[a-z0-9_]`. An app stored as `My-App` was listed as
`starlark:My-App` and looked up as `my_app`, so toggling an app the page
had drawn a moment earlier answered 404. Keys written by
`_install_star_file` are already sanitised, so this only shows up for
manifests written by the starlark-apps plugin itself or edited by hand.

`_validate_starlark_app_path` rejects traversal without rewriting, so it
is the check to use here -- listing and toggling now agree on one key.
The updater also uses `setdefault` rather than indexing: the app is
loaded but its on-disk entry need not exist, and `_update_manifest_safe`
does not catch `KeyError`, so that escaped as a 500 rather than writing
the entry.

**`star_file` was stored absolute.** Readers join it to the app's own
directory -- `_standalone_render_starlark_app` does `app_dir /
app_data.get('star_file', f'{app_id}.star')` -- so the key's default is a
bare filename and an absolute value gave it a second meaning. Since
`Path.__truediv__` discards the left side when the right is absolute,
the manifest was pinned to whatever PROJECT_ROOT installed it, and a
moved or redeployed install could not find its own file. Storing
`dest.name` matches the default and stays relocatable. Read paths are
unchanged, so manifests already holding an absolute path keep working.

7 new tests. Whole suite: no new failures against main, 4013 passed
against 4007.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 16:35:56 -04:00
Chuck 50258635a8 fix(starlark): restore the API routes #330 dropped (#535)
The Pixlet install button reported "Pixlet install failed: Resource not found" -- Flask's 404 handler, because the route did not exist. #253 added thirteen Starlark routes; #330 rewrote api_v3.py and dropped all of them, along with the `starlark:<app_id>` entries that surface installed apps in the plugins list and the toggle branch that enables them.

Restores all thirteen routes, the plugin-list entries and the toggle path, so Pixlet installs, the app store browses and installs, and an installed app can be managed like any other plugin.

Not a straight revert. Three error paths stopped returning exception text to the caller; the manifest write moved off a shared temp filename that two concurrent writers could interleave; both dynamic importers stopped leaving half-initialised modules in sys.modules; the config update rolls back when the save fails; the toggle checks that persistence succeeded; and the path check returns the validated path instead of a boolean so callers stop re-joining the raw value. New tests no longer reach GitHub.

Verified on a 256x64 Pi: Pixlet installs and runs (v0.53.1), the store lists 1000 apps, install/toggle/uninstall round-trip, and traversal and command-injection probes are rejected at every entry.

25 CodeQL alerts dismissed as verified false positives -- path-injection where traversal is blocked, and one list-form subprocess with no shell. Both classes already present on main.

Full core suite: 3981 passed.
2026-09-07 16:06:09 -04:00
ChuckandClaude Opus 5 c9289e3a1d fix(store): update_plugin silently did nothing for four installed plugins (#536)
install_plugin() deliberately renames a plugin's directory to the MANIFEST id
when it differs from the REGISTRY id, so registry `stocks` lands in
`ledmatrix-stocks/`. Every lookup in _find_plugin_path() is by directory name,
so update_plugin("stocks") found nothing, logged "Plugin not installed", and
returned False.

Nothing surfaced that to the user. Clicking update in the web UI was a no-op
with no error, and the plugin stayed on a stale version indefinitely. Four
installed plugins hit this on a real device -- leaderboard, music, stocks and
weather -- found because a scripted update of eleven plugins failed on exactly
those four.

Adds a manifest-id scan as the LAST step of the resolution chain, so the two
documented lookups above it (configured dir, then the sibling plugins/
fallback) keep their exact meaning and ordering. That ordering is pinned by
test_discovery_path_contract.py, which characterises the divergence between
the three resolvers on purpose; this extends the chain rather than reordering
it. Directories renamed aside with '.standalone-backup-' during an install or
rollback are skipped, since matching one would report a half-finished install
as a live plugin.

Also adds scripts/audit_render_path.py, which walks the call graph from
display() and reports blocking calls reachable from it. display() runs on the
render thread, so anything slow there stalls the panel; on a vsync-paced loop
a single 15ms call drops a frame and a network round trip freezes the marquee.
Two instances were already found the slow way, by reading frame-time
histograms -- odds-ticker reading the scoreboard cache per frame, and
soccer-scoreboard timing out inside update(). The audit finds that shape in
the source instead. It is a heuristic and says so: a hit behind an interval
check may be fine.

It currently flags 23 calls across six plugins. The clearest is
ledmatrix-music, whose display() falls back to an inline
requests.get(timeout=5) when album art has not been prefetched -- a deliberate
"show the art rather than go blank" tradeoff by its author, but up to five
seconds of frozen panel. Reported, not changed; that is its owner's call.

185 store tests pass. Three of the six new tests fail without the fix.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 15:55:46 -04:00
5 changed files with 1877 additions and 3 deletions
+159
View File
@@ -0,0 +1,159 @@
#!/usr/bin/env python3
"""Find blocking work reachable from a plugin's render path.
`display()` runs on the render thread. Anything slow reached from it stalls the
panel for its whole duration, and on a vsync-paced loop that is immediately
visible: a single 15ms call on a 100Hz panel drops a frame, and a network round
trip freezes the marquee outright.
This has bitten twice already. odds-ticker called `_has_live_games()` every
frame, whose slow path read the scoreboard cache from disk and parsed JSON per
enabled league -- one stalled frame every few minutes. soccer-scoreboard timed
out inside `update()` during a cache refresh. Both were found by staring at
frame-time histograms, which is a slow way to find a bug that is visible in the
source.
The audit walks the call graph from `display()` through same-class `self.*`
methods and reports anything that reaches a known-blocking API. It is a
heuristic, not a proof: it cannot see through indirection, and a hit is not
automatically a bug -- a call guarded by an interval check may be fine. It is a
list of places worth a human look.
python3 scripts/audit_render_path.py # all plugins
python3 scripts/audit_render_path.py --dir plugin-repos # a specific tree
python3 scripts/audit_render_path.py --plugin odds-ticker
"""
from __future__ import annotations
import argparse
import ast
import sys
from pathlib import Path
#: Calls that can block for longer than a frame. Matched on the attribute or
#: function name, so `requests.get`, `self.session.get` and a bare `get` on a
#: requests-ish object all register.
BLOCKING = {
"get": "network or cache read",
"post": "network",
"put": "network",
"request": "network",
"urlopen": "network",
"read": "I/O",
"open": "file I/O",
"load": "JSON/file parse",
"loads": "JSON parse",
"dump": "file write",
"dumps": "serialise",
"sleep": "sleep on the render thread",
"run": "subprocess",
"check_output": "subprocess",
"connect": "network",
"download_logo": "network",
"_fetch": "fetch",
}
#: Names that make a hit far more likely to matter.
HIGH_SIGNAL = ("requests", "urllib", "session", "cache_manager", "subprocess",
"socket", "http")
RENDER_ENTRY = "display"
class Analyzer:
def __init__(self, tree: ast.AST):
self.methods: dict[str, ast.FunctionDef] = {}
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
self.methods.setdefault(node.name, node)
def calls_in(self, fn: ast.AST):
"""(self-method names called, blocking hits) inside one function."""
self_calls, hits = set(), []
for node in ast.walk(fn):
if not isinstance(node, ast.Call):
continue
func = node.func
if isinstance(func, ast.Attribute):
name = func.attr
base = ast.unparse(func.value) if hasattr(ast, "unparse") else ""
if base == "self" and name in self.methods:
self_calls.add(name)
continue
if name in BLOCKING:
hits.append((name, base, BLOCKING[name], node.lineno))
elif isinstance(func, ast.Name) and func.id in BLOCKING:
hits.append((func.id, "", BLOCKING[func.id], node.lineno))
return self_calls, hits
def reachable_from(self, entry: str, max_depth: int = 3):
"""Blocking hits reachable from `entry`, with the path that reaches them."""
if entry not in self.methods:
return []
found, seen = [], set()
stack = [(entry, [entry], 0)]
while stack:
name, path, depth = stack.pop()
if name in seen or depth > max_depth:
continue
seen.add(name)
self_calls, hits = self.calls_in(self.methods[name])
for hit in hits:
found.append((path, hit))
for callee in sorted(self_calls):
stack.append((callee, path + [callee], depth + 1))
return found
def audit_file(path: Path):
try:
tree = ast.parse(path.read_text(encoding="utf-8"))
except (OSError, SyntaxError):
return []
analyzer = Analyzer(tree)
return analyzer.reachable_from(RENDER_ENTRY)
def main():
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
root = Path(__file__).resolve().parent.parent
ap.add_argument("--dir", default=str(root / "plugin-repos"))
ap.add_argument("--plugin", help="only this plugin directory")
ap.add_argument("--all-hits", action="store_true",
help="include low-signal hits (open/read/load on locals)")
args = ap.parse_args()
base = Path(args.dir)
if not base.is_dir():
sys.exit("not a directory: %s" % base)
plugins = [base / args.plugin] if args.plugin else sorted(
d for d in base.iterdir() if d.is_dir())
total = 0
for plugin in plugins:
rows = []
for src in sorted(plugin.glob("*.py")):
if src.name.startswith("test_"):
continue
for path, (name, s_base, why, lineno) in audit_file(src):
signal = any(h in s_base.lower() for h in HIGH_SIGNAL)
if not signal and not args.all_hits:
continue
rows.append((src.name, lineno, "->".join(path),
("%s.%s" % (s_base, name)) if s_base else name, why))
if rows:
total += len(rows)
print("\n%s" % plugin.name)
for fname, lineno, chain, call, why in sorted(rows):
print(" %s:%-5d %-34s via %s" % (fname, lineno, call + " (" + why + ")", chain))
print("\n%d blocking call(s) reachable from display() across %d plugin(s)"
% (total, len(plugins)))
print("Heuristic: a hit guarded by an interval check may be fine. Look, do "
"not assume.")
if __name__ == "__main__":
main()
+69
View File
@@ -1589,6 +1589,7 @@ class PluginStoreManager:
with open(manifest_path, 'r') as f: with open(manifest_path, 'r') as f:
manifest = json.load(f) manifest = json.load(f)
requested_id = plugin_id
plugin_id = plugin_id or manifest.get('id') plugin_id = plugin_id or manifest.get('id')
if not plugin_id: if not plugin_id:
return { return {
@@ -1664,6 +1665,15 @@ class PluginStoreManager:
branch_info = f" (branch: {branch_used})" if branch_used else "" branch_info = f" (branch: {branch_used})" if branch_used else ""
self.logger.info(f"Successfully installed plugin from URL: {plugin_id}{branch_info}") self.logger.info(f"Successfully installed plugin from URL: {plugin_id}{branch_info}")
# User deliberately (re)installed this plugin -- clear any persistent
# uninstall record, exactly as install_plugin() does. Without this the
# id stays in config/uninstalled_plugins.json and
# purge_uninstalled_plugins(), which runs at every web-app startup,
# deletes the directory again: the plugin works for the rest of the
# session and is gone after the next reboot.
self.forget_uninstalled_plugin(
*(pid for pid in (requested_id, plugin_id, manifest.get('id')) if pid)
)
result = { result = {
'success': True, 'success': True,
'plugin_id': plugin_id, 'plugin_id': plugin_id,
@@ -2424,6 +2434,65 @@ class PluginStoreManager:
except (OSError, ValueError): except (OSError, ValueError):
pass pass
# Last resort: the directory name may differ from the id being looked
# up. install_plugin() deliberately renames a plugin's directory to the
# MANIFEST id when it differs from the REGISTRY id (see the rename near
# "doesn't match registry ID" above), so `stocks` in the registry lands
# in `ledmatrix-stocks/`. Every lookup above is by directory name, so
# update_plugin("stocks") found nothing and reported the plugin as not
# installed -- silently, and for good: the user sees no error and stays
# on a stale version. Four installed plugins hit this in practice
# (leaderboard, music, stocks, weather).
#
# Deliberately last so the two lookups above keep their exact meaning;
# this only runs when a direct hit already failed. See
# test_discovery_path_contract.py, which pins that ordering.
for search_dir in self._candidate_plugin_dirs():
match = self._find_by_manifest_id(search_dir, plugin_id)
if match is not None:
self.logger.debug(
"Resolved plugin '%s' to %s via its manifest id "
"(directory name differs from the id)", plugin_id, match)
return match
return None
def _candidate_plugin_dirs(self) -> List[Path]:
"""Directories that may hold installed plugins, configured one first."""
dirs = [self.plugins_dir]
try:
base = self.plugins_dir if self.plugins_dir.is_absolute() else self.plugins_dir.resolve()
sibling = base.parent / 'plugins'
if sibling != self.plugins_dir:
dirs.append(sibling)
except (OSError, ValueError):
pass
return [d for d in dirs if d.exists()]
@staticmethod
def _find_by_manifest_id(search_dir: Path, plugin_id: str) -> Optional[Path]:
"""A subdirectory of `search_dir` whose manifest declares `plugin_id`.
Skips half-finished installs: store_manager renames a directory aside
with '.standalone-backup-' during install and rollback, and treating
one as installed would resurrect a ghost plugin.
"""
try:
entries = sorted(search_dir.iterdir())
except (OSError, ValueError):
return None
for entry in entries:
if not entry.is_dir() or '.standalone-backup-' in entry.name:
continue
manifest = entry / 'manifest.json'
if not manifest.is_file():
continue
try:
with open(manifest, 'r', encoding='utf-8') as handle:
if json.load(handle).get('id') == plugin_id:
return entry
except (OSError, ValueError):
continue
return None return None
_SKIN_ID_PATTERN = re.compile(r'^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$') _SKIN_ID_PATTERN = re.compile(r'^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$')
+71
View File
@@ -116,6 +116,77 @@ class TestFallbackDivergence:
plugin_dir / "config_schema.json" plugin_dir / "config_schema.json"
class TestRegistryIdVersusDirectoryName:
"""install_plugin() renames a plugin's directory to the MANIFEST id when it
differs from the REGISTRY id, so `stocks` in the registry lands in
`ledmatrix-stocks/`. Every path lookup is by directory name, so
update_plugin("stocks") used to find nothing and report the plugin as not
installed -- silently. Four installed plugins hit this in practice
(leaderboard, music, stocks, weather): the user clicks update, nothing
happens, no error, and they stay on a stale version indefinitely.
"""
def test_resolves_a_plugin_whose_directory_name_differs_from_its_id(self, tmp_path):
configured = tmp_path / "plugin-repos"
configured.mkdir()
plugin_dir = _write_plugin(configured, "stocks", dir_name="ledmatrix-stocks")
store = PluginStoreManager(plugins_dir=str(configured))
assert store._find_plugin_path("stocks") == plugin_dir
def test_direct_directory_hit_still_wins(self, tmp_path):
"""The manifest scan is a last resort. A directory named for the id
must still be preferred, so the two documented lookups above keep
their exact meaning."""
configured = tmp_path / "plugin-repos"
configured.mkdir()
direct = _write_plugin(configured, "demo", dir_name="demo")
# A second directory whose manifest claims the same id.
_write_plugin(configured, "demo", dir_name="zz-other-demo")
store = PluginStoreManager(plugins_dir=str(configured))
assert store._find_plugin_path("demo") == direct
def test_unknown_id_is_still_not_found(self, tmp_path):
configured = tmp_path / "plugin-repos"
configured.mkdir()
_write_plugin(configured, "stocks", dir_name="ledmatrix-stocks")
store = PluginStoreManager(plugins_dir=str(configured))
assert store._find_plugin_path("no-such-plugin") is None
def test_half_finished_installs_are_not_resurrected(self, tmp_path):
"""A directory renamed aside during install/rollback carries a valid
manifest. Matching one would report a ghost plugin as installed."""
configured = tmp_path / "plugin-repos"
configured.mkdir()
_write_plugin(configured, "ghost",
dir_name="ghost.standalone-backup-1234")
store = PluginStoreManager(plugins_dir=str(configured))
assert store._find_plugin_path("ghost") is None
def test_unreadable_manifest_does_not_break_the_scan(self, tmp_path):
configured = tmp_path / "plugin-repos"
configured.mkdir()
broken = configured / "broken-plugin"
broken.mkdir()
(broken / "manifest.json").write_text("{ not json")
wanted = _write_plugin(configured, "stocks", dir_name="ledmatrix-stocks")
store = PluginStoreManager(plugins_dir=str(configured))
assert store._find_plugin_path("stocks") == wanted
def test_sibling_plugins_dir_is_searched_too(self, tmp_path):
configured = tmp_path / "plugin-repos"
configured.mkdir()
legacy = _write_plugin(tmp_path / "plugins", "weather",
dir_name="ledmatrix-weather")
store = PluginStoreManager(plugins_dir=str(configured))
assert store._find_plugin_path("weather") == legacy
class TestStandaloneBackupContract: class TestStandaloneBackupContract:
def test_discovery_skips_backup_dirs(self, tmp_path): def test_discovery_skips_backup_dirs(self, tmp_path):
plugins_dir = tmp_path / "plugins" plugins_dir = tmp_path / "plugins"
@@ -0,0 +1,395 @@
"""The Starlark routes the frontend calls must exist.
`plugins_manager.js` posts to /api/v3/starlark/install-pixlet and then reloads
/api/v3/starlark/status. Neither route existed: #330 rewrote api_v3.py and
dropped all thirteen Starlark routes that #253 had added, so both calls fell
through to Flask's 404 handler, which answers
{"status": "error", "message": "Resource not found"}
and the button reported "Pixlet install failed: Resource not found" -- a
message that names neither the resource nor the cause.
These assert the routes are registered and answer in the shape the frontend
reads, so a future rewrite of this file cannot silently drop them again.
"""
import json
import os
from unittest.mock import MagicMock, patch
import pytest
@pytest.fixture
def client():
from web_interface.app import app
app.config['TESTING'] = True
with app.test_client() as c:
yield c
class TestRoutesAreRegistered:
"""The failure was a missing route, so check the URL map directly.
All thirteen, not just the two the Pixlet button needs: #330 dropped the
lot, and the app store page is built on repository/browse,
repository/categories and repository/install, which 404 the same way.
"""
@pytest.mark.parametrize("rule,method", [
("/api/v3/starlark/install-pixlet", "POST"),
("/api/v3/starlark/status", "GET"),
("/api/v3/starlark/apps", "GET"),
("/api/v3/starlark/upload", "POST"),
("/api/v3/starlark/repository/browse", "GET"),
("/api/v3/starlark/repository/categories", "GET"),
("/api/v3/starlark/repository/install", "POST"),
("/api/v3/starlark/apps/<app_id>", "GET"),
("/api/v3/starlark/apps/<app_id>", "DELETE"),
("/api/v3/starlark/apps/<app_id>/config", "GET"),
("/api/v3/starlark/apps/<app_id>/config", "PUT"),
("/api/v3/starlark/apps/<app_id>/toggle", "POST"),
("/api/v3/starlark/apps/<app_id>/render", "POST"),
])
def test_route_exists(self, rule, method):
from web_interface.app import app
matches = [r for r in app.url_map.iter_rules()
if r.rule == rule and method in r.methods]
assert matches, (
f"{method} {rule} is not registered; the frontend calls it and "
"would get Flask's generic 'Resource not found'")
class TestInstallPixlet:
def test_it_does_not_404(self, client):
with patch('web_interface.blueprints.api_v3.subprocess.run') as run:
run.return_value = MagicMock(returncode=0, stdout="ok", stderr="")
resp = client.post('/api/v3/starlark/install-pixlet')
assert resp.status_code != 404, "the route is still missing"
assert resp.get_json().get('message') != 'Resource not found'
def test_success_is_reported_in_the_shape_the_button_reads(self, client):
with patch('web_interface.blueprints.api_v3.subprocess.run') as run:
run.return_value = MagicMock(returncode=0, stdout="done", stderr="")
resp = client.post('/api/v3/starlark/install-pixlet')
body = resp.get_json()
assert body['status'] == 'success', body
assert 'message' in body, "the JS shows data.message on success"
def test_a_failed_download_says_why(self, client):
with patch('web_interface.blueprints.api_v3.subprocess.run') as run:
run.return_value = MagicMock(returncode=1, stdout="", stderr="no such release")
resp = client.post('/api/v3/starlark/install-pixlet')
body = resp.get_json()
assert body['status'] == 'error'
assert 'no such release' in body['message'], \
"the installer's own stderr is what tells the user what went wrong"
def test_a_timeout_is_reported_rather_than_hanging(self, client):
import subprocess as sp
with patch('web_interface.blueprints.api_v3.subprocess.run',
side_effect=sp.TimeoutExpired(cmd='x', timeout=300)):
resp = client.post('/api/v3/starlark/install-pixlet')
assert resp.get_json()['status'] == 'error'
assert 'timed out' in resp.get_json()['message'].lower()
class TestStarlarkStatus:
def test_it_does_not_404(self, client):
resp = client.get('/api/v3/starlark/status')
assert resp.status_code != 404, "the route is still missing"
assert resp.get_json().get('message') != 'Resource not found'
def test_it_reports_pixlet_availability_without_the_plugin_loaded(self, client):
# The status call runs before install too -- it must answer even when
# starlark-apps is not loaded, which is the state a user is in when
# they press the install button for the first time.
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None):
resp = client.get('/api/v3/starlark/status')
body = resp.get_json()
assert body['status'] == 'success', body
assert 'pixlet_available' in body
assert body['plugin_loaded'] is False
class TestTheInstallerScriptIsActuallyThere:
def test_download_pixlet_script_exists_and_is_executable(self):
# install_pixlet chmods and runs this; a missing file is the one error
# it reports as a 404 of its own, which would look identical to the
# bug being fixed here.
import os
from pathlib import Path
from web_interface.blueprints.api_v3 import PROJECT_ROOT
script = Path(PROJECT_ROOT) / 'scripts' / 'download_pixlet.sh'
assert script.is_file(), f"{script} is missing; install_pixlet would 404"
assert os.access(script, os.R_OK)
class TestTheAppStoreFlow:
"""Browsing and installing from the Tronbyte repository.
These are the calls the app store page makes. Each returned the generic
"Resource not found" before this change, which is indistinguishable from
an empty store.
"""
@pytest.fixture
def offline_repo(self):
"""No live GitHub calls from the test suite.
browse and categories reach _get_tronbyte_repository_class() and then
list_all_apps_cached(); with a cold server-side cache that is a real
network request, which makes the run slow, rate-limitable, and able to
pass on a 500 because these assertions only check for a 404.
"""
repo = MagicMock()
# Matches what the real list_all_apps_cached returns; the handler
# indexes every one of these keys.
repo.return_value.list_all_apps_cached.return_value = {
'apps': [{'id': 'quoteoftheday', 'name': 'A Quote A Day',
'category': 'text'}],
'categories': ['text'],
'authors': ['someone'],
'count': 1,
'cached': True,
}
repo.return_value.get_rate_limit_info.return_value = {'remaining': 5000}
with patch('web_interface.blueprints.api_v3._get_tronbyte_repository_class',
return_value=repo):
yield repo
def test_browse_does_not_404(self, client, offline_repo):
resp = client.get('/api/v3/starlark/repository/browse')
assert resp.status_code != 404, "the store cannot list anything"
assert resp.get_json().get('message') != 'Resource not found'
def test_browse_returns_the_apps_the_store_lists(self, client, offline_repo):
resp = client.get('/api/v3/starlark/repository/browse')
body = resp.get_json()
assert body['status'] == 'success', body
assert any(a.get('id') == 'quoteoftheday' for a in body.get('apps', [])), body
def test_categories_does_not_404(self, client, offline_repo):
resp = client.get('/api/v3/starlark/repository/categories')
assert resp.status_code != 404
assert resp.get_json().get('message') != 'Resource not found'
def test_no_live_network_call_is_made(self, client, offline_repo):
client.get('/api/v3/starlark/repository/browse')
assert offline_repo.called, \
"the route did not go through the patched repository class"
def test_installed_apps_list_does_not_404(self, client):
resp = client.get('/api/v3/starlark/apps')
assert resp.status_code != 404
assert resp.get_json().get('message') != 'Resource not found'
def test_repository_install_rejects_a_missing_body_rather_than_404ing(self, client):
# A 400/422 here is the route working: it received the call and said
# what was wrong. A 404 means it was never reached at all.
resp = client.post('/api/v3/starlark/repository/install',
json={}, content_type='application/json')
assert resp.status_code != 404, "the install route is still missing"
assert resp.get_json().get('message') != 'Resource not found'
def test_upload_rejects_an_empty_post_rather_than_404ing(self, client):
resp = client.post('/api/v3/starlark/upload')
assert resp.status_code != 404
assert resp.get_json().get('message') != 'Resource not found'
class TestNoStarlarkRouteIsMissing:
"""A single check that the whole set is present.
#330 removed all thirteen at once by rewriting this file. One assertion
over the frontend's own list is what would have caught that.
"""
def test_every_endpoint_the_frontend_calls_is_registered(self):
import re
from pathlib import Path
from werkzeug.exceptions import MethodNotAllowed, NotFound
from web_interface.app import app
root = Path(__file__).resolve().parent.parent.parent
js = (root / 'web_interface' / 'static' / 'v3' / 'plugins_manager.js').read_text()
# The frontend builds some of these with template literals, e.g.
# `/api/v3/starlark/apps/${appId}/toggle`. Substitute a placeholder so
# the URL is concrete, then let Werkzeug match it the way a request
# would -- string comparison cannot see <app_id> rules.
raw = set(re.findall(r"[\'\"`](/api/v3/starlark/[^\'\"`\s]*)", js))
urls = set()
for u in raw:
u = re.sub(r"\$\{[^}]*\}", "probe", u)
urls.add(u.rstrip('/') or u)
assert urls, "found no starlark calls in the frontend -- did the file move?"
adapter = app.url_map.bind('localhost')
missing = []
for u in sorted(urls):
try:
adapter.match(u, method='GET')
except MethodNotAllowed:
pass # route exists, just not for GET -- fine
except NotFound:
missing.append(u)
assert not missing, f"the frontend calls these and they are not registered: {missing}"
class TestInstalledAppsAppearWithTheOtherPlugins:
"""An installed .star app must be manageable like any other plugin.
#253 surfaced installed apps in /plugins/installed as `starlark:<app_id>`
entries, so they could be seen and enabled/disabled from the same list as
everything else, and routed `starlark:` toggles to the Starlark manifest.
#330 removed both. The result was an app that installs successfully, then
appears nowhere and cannot be turned on or off.
"""
APPS = {'apps': {'quoteoftheday': {'name': 'A Quote A Day', 'enabled': True}}}
def test_an_installed_app_is_listed(self, client):
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None), \
patch('web_interface.blueprints.api_v3._read_starlark_manifest', return_value=self.APPS):
resp = client.get('/api/v3/plugins/installed')
ids = [p['id'] for p in resp.get_json()['data']['plugins']]
assert 'starlark:quoteoftheday' in ids, \
"an installed Starlark app does not appear among the plugins"
def test_the_entry_carries_what_the_ui_needs(self, client):
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None), \
patch('web_interface.blueprints.api_v3._read_starlark_manifest', return_value=self.APPS):
resp = client.get('/api/v3/plugins/installed')
entry = next(p for p in resp.get_json()['data']['plugins']
if p['id'] == 'starlark:quoteoftheday')
assert entry['name'] == 'A Quote A Day'
assert entry['enabled'] is True
assert entry['is_starlark_app'] is True, "the UI keys its Starlark handling off this"
assert entry['category'] == 'Starlark App'
def test_a_starlark_failure_does_not_empty_the_plugin_list(self, client):
# The virtual entries are appended to the real ones; a broken manifest
# must cost the Starlark rows, not everybody else's.
with patch('web_interface.blueprints.api_v3._get_starlark_plugin',
side_effect=RuntimeError('boom')):
resp = client.get('/api/v3/plugins/installed')
assert resp.status_code == 200
assert resp.get_json()['status'] == 'success'
def test_toggling_an_app_does_not_report_plugin_not_found(self, client):
written = {}
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None), \
patch('web_interface.blueprints.api_v3._read_starlark_manifest',
return_value={'apps': {'quoteoftheday': {'enabled': True}}}), \
patch('web_interface.blueprints.api_v3._write_starlark_manifest',
side_effect=lambda m: written.update(m) or True):
resp = client.post('/api/v3/plugins/toggle',
json={'plugin_id': 'starlark:quoteoftheday', 'enabled': False})
body = resp.get_json()
assert body['status'] == 'success', body
assert body['enabled'] is False
assert written['apps']['quoteoftheday']['enabled'] is False, \
"the manifest was not actually updated"
def test_toggling_an_unknown_app_says_so(self, client):
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None), \
patch('web_interface.blueprints.api_v3._read_starlark_manifest',
return_value={'apps': {}}):
resp = client.post('/api/v3/plugins/toggle',
json={'plugin_id': 'starlark:nope', 'enabled': True})
assert resp.status_code == 404
assert 'nope' in resp.get_json()['message']
def test_a_traversal_app_id_is_rejected_before_touching_the_manifest(self, client):
resp = client.post('/api/v3/plugins/toggle',
json={'plugin_id': 'starlark:../../etc/passwd', 'enabled': True})
assert resp.status_code == 400
assert 'traversal' in resp.get_json()['message']
def test_an_app_id_the_listing_published_can_be_toggled(self, client):
"""The id here is exactly what _starlark_virtual_plugins publishes.
It used to be re-slugified on the way back in -- lowercased, with every
character outside [a-z0-9_] replaced -- so an app stored as 'My-App'
was listed as 'starlark:My-App' and looked up as 'my_app'. Toggling an
app the page had just drawn answered 404.
"""
manifest = {'apps': {'My-App': {'name': 'My App', 'enabled': False}}}
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None), \
patch('web_interface.blueprints.api_v3._read_starlark_manifest',
return_value=manifest), \
patch('web_interface.blueprints.api_v3._write_starlark_manifest',
return_value=True) as write:
resp = client.post('/api/v3/plugins/toggle',
json={'plugin_id': 'starlark:My-App', 'enabled': True})
assert resp.status_code == 200, resp.get_json()
assert write.called, "the toggle never reached the manifest"
assert manifest['apps']['My-App']['enabled'] is True
def test_a_loaded_app_missing_from_the_manifest_does_not_500(self, client):
"""_update_manifest_safe does not catch KeyError, so indexing an entry
that is not on disk yet escaped as a 500 instead of writing it."""
app = MagicMock()
app.manifest = {'enabled': False}
plugin = MagicMock()
plugin.apps = {'demo': app}
written = {}
def run_updater(fn):
fn(written)
return True
plugin._update_manifest_safe.side_effect = run_updater
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=plugin):
resp = client.post('/api/v3/plugins/toggle',
json={'plugin_id': 'starlark:demo', 'enabled': True})
assert resp.status_code == 200, resp.get_json()
assert written['apps']['demo']['enabled'] is True
class TestTheManifestStaysRelocatable:
"""`star_file` is joined to the app's own directory by its readers.
_standalone_render_starlark_app does `app_dir / app_data.get('star_file',
f'{app_id}.star')`, so the key's default is a bare filename. Storing an
absolute path gave it a second meaning, and Path.__truediv__ discards the
left side when the right is absolute -- which pinned the manifest to the
PROJECT_ROOT that installed it.
"""
@pytest.fixture
def starlark_dir(self, tmp_path, monkeypatch):
from web_interface.blueprints import api_v3 as module
apps_dir = tmp_path / "starlark-apps"
apps_dir.mkdir()
monkeypatch.setattr(module, '_STARLARK_APPS_DIR', apps_dir)
monkeypatch.setattr(module, '_STARLARK_MANIFEST_FILE', apps_dir / 'manifest.json')
return apps_dir
def _install(self, tmp_path):
from web_interface.blueprints import api_v3 as module
source = tmp_path / "source.star"
source.write_text("# app")
with patch.object(module, '_get_pixlet_renderer_class',
side_effect=ImportError("no pixlet here")):
assert module._install_star_file('demo', str(source), {'name': 'Demo'})
return json.loads((module._STARLARK_MANIFEST_FILE).read_text())['apps']['demo']
def test_the_star_file_is_recorded_by_name(self, starlark_dir, tmp_path):
assert self._install(tmp_path)['star_file'] == 'demo.star'
def test_the_recorded_path_is_not_absolute(self, starlark_dir, tmp_path):
"""An absolute value survives a move only by accident."""
assert not os.path.isabs(self._install(tmp_path)['star_file'])
def test_the_value_resolves_against_the_app_directory(self, starlark_dir, tmp_path):
"""Which is the one thing every reader of this key does with it."""
entry = self._install(tmp_path)
assert (starlark_dir / 'demo' / entry['star_file']).is_file()
def test_it_matches_the_default_a_reader_falls_back_to(self, starlark_dir, tmp_path):
"""Stored and defaulted values must mean the same thing."""
assert self._install(tmp_path)['star_file'] == 'demo.star'
File diff suppressed because it is too large Load Diff