Compare commits

..
15 Commits
Author SHA1 Message Date
ChuckBuildsandClaude Opus 5 37fc1b56b5 fix(composer): coerce prefixed colour channels, non-finite numbers, marquee ids
Three more routes into the generated source, plus a fix to one of my own tests
that was checking the wrong branch.

Prefixed colour channels were interpolated raw
----------------------------------------------
Five tuples were built without coercion:

    p['fill_tuple']  = f"({el.get('r', 100)}, {el.get('g', 200)}, ...)"
    p['empty_tuple'] = f"({el.get('emptyR', 50)}, ...)"
    p['label_tuple'] = f"({el.get('labelR', 200)}, ...)"

so progress_bar, pips, sparkline and gauge took arbitrary expressions the same
way width/height did. Confirmed: every one of the five put __import__ into the
generated source. They now go through a new _rgb_tuple helper, which _rgb_expr
also delegates to.

The pre-existing colour test only covered r/g/b on a text element, which is why
the prefixed channels and these four types were never exercised.

Non-finite numbers escaped as a 500
-----------------------------------
json.loads accepts Infinity/-Infinity/NaN by default and Flask's get_json
passes them straight through, so a payload can hand _safe_int a non-finite
float. int(inf) raises OverflowError, which is neither ValueError nor
ComposerInputError, so it escaped both handlers and surfaced as a 500 with a
traceback rather than a 422. Verified end to end through Flask's parser.

Marquee ids reached the source as identifiers
---------------------------------------------
data_key is spliced UNQUOTED into variable names (_{{ data_key }}_text = ...)
and only '-' was normalised. A punctuated id landed in the generated source as
code. ast.parse caught it, so this was not exploitable, but the caller got an
opaque "Generated code has a syntax error" instead of being told the id was
unusable -- the same failure mode as the empty-block bug. Now restricted to
identifier characters and bounded to 64.

The line-anchor test was testing the wrong branch
-------------------------------------------------
test_line_branch_applies_the_anchor_offset searched the whole file for
"case 'line': {". getBoundingBox has one too and comes first, so the assertion
was reading the bounding-box branch: stripping the anchor offset from
_drawElement left all 11 checks green. Both line tests are now scoped to their
own function via tree-sitter, so they cannot be satisfied by the same branch.

Tests: 35 of the injection suite's checks fail against the reverted fixes; the
scoped line test fails when _drawElement's offset is removed. Full suite 4059
passed, the one failure being test_install_lowmem (pre-existing, awaiting #492).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-22 14:35:30 -04:00
ChuckBuildsandClaude Opus 5 acc55ef119 fix(composer): scale strokes, anchor lines, snapshot state changes
Three review findings in the composer's JavaScript, all confirmed against the
code.

Stroke widths did not scale with SCALE
--------------------------------------
_drawElement scales all geometry by `s`, but left ctx.lineWidth in canvas
pixels, so at SCALE>1 every outline rendered thinner than one LED pixel and
the preview stopped matching the panel it is previewing. Fixed for rectangle,
ellipse, arc, rounded_rectangle, line, divider and progress_bar. Ellipse and
arc also inset their radii by half the scaled width -- a stroke straddles its
path, so without the inset the outline spills outside the element's bounds.
The gauge branch already did this; the rest now match it.

Selection handles and the grid stay in canvas pixels deliberately: they are
editor chrome, not LED geometry, and live in other functions.

`line` ignored anchors
----------------------
_drawElement resolves ax/ay for every element, but the line branch drew raw
el.x0/el.y0/el.x1/el.y1. Setting xAnchor or yAnchor moved every other element
type and left lines where they were. getBoundingBox had the same omission, so
even once a line moved its hit box would not have. Both now translate by
(ax - el.x0, ay - el.y0); ax resolves from el.x0 for a line, so that is
exactly the anchor offset.

Four state mutations skipped _snapshot
--------------------------------------
_snapshot serialises metadata and currentPreset and is the only caller of
_debouncedAutosave. onBgColorChange, setCustomSize, changePreset and
applyPresetLabel each changed exactly those values without calling it, so the
background colour and the canvas size were lost on reload and could not be
undone. Same defect already fixed in onColorChange.

The review named three; applyPresetLabel has it too -- it is the branch that
handles sizes absent from DISPLAY_PRESETS.

Snapshotting is on the user-driven path only. _applyState and loadTemplate
drive these with {silent: true} while restoring, and snapshotting there would
push restore steps onto the undo stack and re-autosave the state just loaded.

Tests
-----
No JS runner here, so test_composer_js_contracts.py asserts on the parse tree
via tree-sitter: both files parse, no bare `ctx.lineWidth = 1` inside
_drawElement, the line branch and its bounding box carry the anchor offset,
each of the five mutations snapshots, and the two preset paths keep their
!opts.silent guard ahead of the snapshot.

9 of its 11 checks fail against the previous JS. Full suite 3978 passed, the
one failure being test_install_lowmem (pre-existing, awaiting #492).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-22 13:52:48 -04:00
ChuckBuildsandClaude Opus 5 1a0864e5d4 fix(composer): clamp width/height before they reach generated source
Code injection, found by chasing why a security test could not have caught it.

_preprocess_elements built the far corner of five shapes by interpolating the
payload's width/height straight into generated Python:

    w = el.get('width', 10)
    p['x2_expr'] = f"({x_expr}) + {w}"

so a rectangle with width='0 or __import__("os").system("id")' generated

    [0, 0, (0) + 0 or __import__("os").system("id"), (0) + 8],

inside a manager.py that /api/install writes to disk and the plugin loader
imports and executes. rectangle, arc, ellipse, rounded_rectangle and gauge all
share the pattern. Both fields now go through _safe_int, like every other
geometry value.

Unreachable today only because composer_bp is still unregistered -- the same
caveat as the docstring injection fixed earlier in this PR.

Why the existing test missed it
-------------------------------
test_a_non_numeric_geometry_value_cannot_reach_the_source drove its payloads
through a "line" element. manager.py.j2 has never had a `line` branch, so
_preprocess_elements produced nothing for it and no value it set could reach
the generated source. Every assertion passed trivially, against code that was
in fact vulnerable. The test has been vacuous since it was written; the
_RENDERABLE_ELEMENT_TYPES constant added in the previous commit only made the
cause legible.

It now runs across the five types that actually render, over x/y/width/height:
40 of those cases fail with the clamping reverted, where the old version
passed 100%.

A second test asserts every type used by the injection suite is in
_RENDERABLE_ELEMENT_TYPES, so the suite cannot quietly go vacuous again.

Also: _payload set "config_vars", but _generate_plugin_files reads
data['dataModel']['configVars']. Nothing passed through that key was ever
read. Fixed so config-var tests exercise the real path.

Full suite: 3967 passed, the one failure being test_install_lowmem
(pre-existing, awaiting #492).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-22 13:11:47 -04:00
ChuckBuildsandClaude Opus 5 732c7d1a30 fix(composer): an element the template cannot draw broke generation
Reproduced from the review comment. A `group` element carrying minWidth
generated:

    if width >= 64:  # breakpoint: 64px+ displays only
    # <nothing>

manager.py.j2 wraps each element in the breakpoint and blink blocks, but the
body comes from the per-type branches -- and a type with no branch contributes
nothing, so the wrapper opens a block with no statements. ast.parse then fails
and the caller is told only "Generated code has a syntax error: expected an
indented block ... line 49", naming a line of generated source they never see.

Two defences:

- _preprocess_elements drops types the template has no branch for, alongside
  the existing `section` skip. This is the root cause: those elements should
  never have reached the template.
- The branch chain ends in `{% else %}pass`, so a type added to the canvas
  before its drawing branch exists degrades to a no-op rather than a plugin
  that will not parse.

The review also cited dynamic_text with binding_source != 'config'. That one
does not reproduce -- the branch emits a draw_text regardless -- which is why
an earlier attempt to reproduce this found nothing.

_RENDERABLE_ELEMENT_TYPES has to stay in step with the template: a type listed
with no branch emits an empty block again, and a branch missing from the list
is silently dropped from every generated plugin. A test asserts the two sets
are equal rather than trusting them to be maintained together.

Tests: 12 new, covering group/unknown/section against breakpoint, blink and
both nested, plus the set-equality and fallback checks. 7 fail with both
defences reverted. 172 composer tests pass; full suite 3862 passed, the one
failure being test_install_lowmem (pre-existing, awaiting #492).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-22 12:16:44 -04:00
ChuckBuildsandClaude Opus 5 d42593e7ce Merge main into feat/plugin-composer, and fix three review findings
The branch was 57 commits behind and conflicting. I had put the rebase
aside earlier as needing the author's eyes, on the grounds that the PR is
+5091 lines -- but that was the wrong measure. The actual conflict was a
single hunk in app.css, where this branch adds .md\:inline and main added
.md\:block and .md\:w-auto at the same place. All three are kept.

Merging rather than rebasing: the branch is public and 57 commits behind,
so a rebase would rewrite shared history for a force-push.

Three findings fixed on top:

A missing `text` or `format` was a 500. `p` is a copy of the raw element
and the defaults were applied to the locals t1/fmt1 only, so an element
omitting either key left it absent, manager.py.j2 rendered
`{{ el.text | tojson }}` over a jinja2.Undefined, and tojson raised
TypeError -- which no handler catches:

    text without 'text':    TypeError: Object of type Undefined is not
                            JSON serializable
    clock without 'format': same

Both keys are now set explicitly. Verified: removing either assignment
fails 4 of the new tests.

E741 on my own injection-test file: two `for i, l in enumerate(...)`
loops, which ruff rejects and would fail a lint-gated build. Renamed.
Ruff now clean on all three files this PR touches.

Not done: registering composer_bp. This PR's own description gates it --
"Not yet wired up ... tracking as a follow-up", with an unchecked box for
"Register composer_bp in app.py before merging or exposing this route" --
so it is a deliberate decision, not an oversight. Confirmed the blueprint
appears in no register_blueprint call outside this branch's tests, which
also means the code-injection fixed earlier in this PR was never
reachable in a deployed instance. Worth fixing before the route is
exposed; not worth exposing the route to satisfy a review comment.

Verified on the merged tree: 3850 passed, 1 failed, 60 skipped. The
failure is test_install_lowmem's tmpfs assumption, which is fixed in #492
and not yet on main. The static audit now passes 3/3 -- the twelve
classes it flagged before were defined on main all along and only looked
missing because this branch was behind.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-21 20:51:14 -04:00
ChuckBuildsandClaude Opus 5 f0bef7784c fix(composer): eight editor bugs from review
All confirmed by reading the code rather than taken on trust.

Saved designs restored onto the wrong canvas. _buildPayload writes the
size as `preset`; _applyState read `state.currentPreset`, which is never
present, so changePreset(undefined) hit its `if (!preset) return` and did
nothing -- silently. A 256x64 design reopened at 128x32 with every
element misplaced. importDesign passed no size key at all, same result.
Both go through a new applyPresetLabel(), which also handles the custom
labels setCustomSize() writes ("200x50"): those are deliberately absent
from DISPLAY_PRESETS, so changePreset alone could never round-trip them.

Keyboard shortcuts hijacked text fields. The `inInput` guard sat below
the Ctrl/Cmd block, under a comment claiming combos "work everywhere".
In any input, Ctrl+C copied the selected *element* -- preventDefault
stopping the real copy -- Ctrl+V pasted an element, Ctrl+A could not
select the field contents, and Tab always moved the element selection,
so keyboard users could not reach the next input. Guard moved above both
blocks, and it now covers contenteditable too.

Resize handles were advertised on five shapes that ignored them. The
canvas drew handles for six element types; the editor gated resize and
hover on `type === 'rectangle'`. The list was also duplicated inside the
canvas. One exported RESIZABLE_TYPES now feeds all four sites.

Lines jumped on drag. addElement assigns x/y *before* spreading
ELEMENT_DEFAULTS, and the line defaults define only x0/y0 -- so a line
carries both, with x at canvas/4 and x0 at 0. Drag and nudge move x0/y0
only, so _getStoredPos preferring `x` handed the drag a base it never
updates.

Colour-picker edits were lost on reload. onColorChange mutated the
element but never set isDirty or called _snapshot, and _debouncedAutosave
only runs from _snapshot. applyPaletteColor did both; they match now.

Also: section elements drew nothing and reported a 0x0 box, so adding
"Section Label" from the palette looked broken and the element was
selectable only through the 3px hit-test padding -- they now draw their
label, with the bounding box using the same font fallback as the draw
call so the two agree. The gauge inset its arc radius by lw/2 where lw is
LED pixels and the radius is canvas pixels, then stroked at lw*s, so the
arc spilled outside its own bounding box at any scale above 1. And the
plugin id is encodeURIComponent'd before it becomes part of a request
path.

Verified: composer-app.js and composer-canvas.js parse cleanly under
tree-sitter (esprima cannot read this codebase -- it predates ??, and
fails identically on the unmodified files). Every symbol referenced
across module boundaries checked to exist. 156 Python composer tests
pass. The static-audit failure is the same 13 classes as before, all
defined on main and absent only because this branch is behind; nothing
here touches CSS or templates.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-21 20:14:52 -04:00
ChuckBuildsandClaude Opus 5 986f74e38b fix(composer): reject config keys that shadow plugin state
Five review findings, plus the two bandit reported.

Config variable keys were checked against an identifier regex only.
Python keywords slipped past it and were caught downstream by ast.parse,
but reported as

    Generated code has a syntax error: invalid syntax (<unknown>, line 17)

which names neither the field nor the value. They are now refused by
name, soft keywords ('match', 'case') included.

Worse, a key matching a BasePlugin attribute generated *valid* code that
silently clobbered plugin state. 'config' is the sharp one: the
assignment lands immediately after super().__init__(), so

    self.config = config.get("config", "x")

replaces the plugin's config dict with a string, and every later
self.config.get(...) fails at runtime. Refused now, along with logger,
display_manager, cache_manager, plugin_id, enabled, self and the
lifecycle method names. A test pins the ordering assumption that reserved
list rests on, so it fails if config vars are ever emitted before
super().__init__() instead.

Also:

- The silent `except Exception: pass` around manifest parsing now logs.
  It left "partial import produced nothing" indistinguishable from a
  malformed manifest. (bandit B110)
- list_plugins() called iterdir() on a directory that may not exist --
  a fresh install or a bad path returned 500 instead of an empty list.
- metadata.id is stripped in the two route handlers, matching
  _generate_plugin_files, which strips before validating. Without it
  " my-plugin " generated fine and then failed the id check at install,
  reading as a generator bug.
- The jinja Environment's autoescape=False now says why: these templates
  emit Python, and escaping a quote to &#34; inside generated code would
  break it. Safety comes from the values instead -- _safe_int, _rgb_expr
  and _reject_source_breaking, all covered by the injection suite.
  (bandit B701, marked nosec with that rationale)

bandit on composer.py: 2 findings -> 0.

Verified: 156 tests across the two composer suites. Removing either new
key check fails 9.

Not reproduced: the suggestion to emit `pass` so a conditional block is
never empty. 'line' and 'divider' render through a different template
branch and 'section' emits nothing at all, so no element type available
here produces an `if width >= N:` with an empty body. Left alone rather
than changing template output speculatively.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-21 19:38:54 -04:00
ChuckBuildsandClaude Opus 5 e450a6dfb6 fix(composer): stop payload text reaching generated Python as code
Review flagged this as critical and it is: the composer builds manager.py
by interpolating payload values into source text, /api/install writes
that file into plugins_dir, and the plugin loader imports and executes
it. The ast.parse check further down rejects only *invalid* syntax, and
an injected `import os` is perfectly valid.

Confirmed against the code before this commit. A plugin name carrying a
triple quote closes the module docstring and everything after it becomes
module-level code:

    generated manager.py parses: True
    injected module-level statements: ['import os', 'PWNED = os.getuid()']

and a geometry value is interpolated verbatim, because the parameter is
annotated int but arrives as JSON:

    _compute_pos_expr('0 or __import__("os").system("id")', 'right', 'width')
      -> 'width - 0 or __import__("os").system("id")'
    generated source: x=0 or __import__("os").system("id"),

Three fixes. _safe_int coerces and optionally clamps, and
_compute_pos_expr applies it to its own argument -- which covers all
twenty-odd call sites at once rather than patching each. _rgb_expr does
the same for the eight colour interpolations, clamping channels to
0-255. Line endpoints and widths go through it too.

For the docstring, _reject_source_breaking refuses a plugin name
containing a quote, backslash or newline. Rejecting rather than escaping:
these are display names, none of that belongs in one, and a clear "Plugin
name cannot contain a double quote." beats silently mangling what the
user typed.

Verified: all three exploits now refused or neutered, and each defence
mutation-checked separately --

    coercion removed in _compute_pos_expr ->  8 failed
    docstring guard removed               ->  5 failed
    colour channels interpolated raw      -> 13 failed

87 tests, covering seven expression payloads across seven geometry
fields and three colour channels, five literal-breaking names, and the
clean case asserting a normal payload still yields no module-level
statements at all.

One aside: the first version of this test file put the exploit string
in its own module docstring, which closed it and made the file a syntax
error -- the same bug, one level up. It now describes the payload rather
than embedding it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-21 19:03:13 -04:00
ChuckBuildsandClaude Opus 5 5133643600 fix(composer): build the font path from the allowlist entry
secure_filename cleared the plugin-directory alerts: CodeQL went from 19
to 5, and from 16 high-severity to 2. The two that remain are in
serve_font, which is gated by a frozenset of three exact filenames -- so
nothing was exploitable -- but the name reaching the filesystem was still
the request value.

It now comes from the matched allowlist entry. Identical strings, so
runtime behaviour is unchanged; the difference is that the filename is
provably a module constant rather than a guarded piece of user input.

The first test I wrote for this proved nothing. It asserted 404 on
traversal payloads, but Flask's router will not match a path segment
containing '/', and the rest 404 simply because no such file exists --
so removing the allowlist entirely still passed. Replaced with a readable
file planted next to the fonts:

    fonts/id_rsa.ttf  ->  404, body does not contain its contents

which fails with "a readable non-allowlisted file was served" the moment
the gate is removed.

45 tests.

Left alone: three medium py/stack-trace-exposure alerts on the
_generate_plugin_files handlers, which return str(exc) for
ComposerInputError. Its seven raise sites are all authored literals
("Author is required.", "Config variable key X is not a valid Python
identifier."), so no traceback or path is exposed. Clearing them means
either replacing that feedback with a generic string or restructuring
validation to return errors instead of raising -- a change to the
author's design, made blind, since CodeQL cannot be run locally to
confirm it would even work. That is a decision, not a cleanup.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-21 18:56:00 -04:00
ChuckBuildsandClaude Opus 5 5929190e36 fix(composer): sanitise the id with the form CodeQL recognises
Previous attempt got the count from 22 down to 19 but left the 16
path-injection alerts untouched: CodeQL carries taint through
_plugin_dir's return value and does not treat an internal realpath /
commonpath guard as a sanitiser.

secure_filename is one it does model. It is also a no-op on every id the
regex accepts -- verified across the accepted alphabet, 4000 generated
ids, zero altered -- so it cannot rewrite a caller's id into a different
plugin's directory. The equality check makes that explicit: if it changes
anything, the id was not one we accept, and we refuse rather than
silently redirect.

Found a real bug while testing the layers separately: '.' resolved to the
plugins root, and install() calls shutil.rmtree(target) when force is
set, so an id of '.' would have deleted every installed plugin. The regex
blocks it today, but the containment layer was allowing candidate == base
on the grounds that the base is not "outside" itself. A plugin directory
must be a child, never the root.

That came out of writing the isolated tests. Removing containment did not
fail anything, because secure_filename rejects traversal first -- which
made a redundant layer look load-bearing. Each layer is now neutralised
in turn so the one under test is the only thing standing:

    containment removed        -> FAIL (13 payloads reach the base or past it)
    candidate == base allowed  -> FAIL ('.' resolves to the plugins root)
    commonpath -> startswith   -> FAIL (sibling "plugins-evil" accepted)
    secure_filename bypassed   -> pass, containment covers it

The last is honest rather than a gap: with containment in place the
sanitiser has nothing left to block, and its value here is CodeQL
recognition plus a second barrier if containment is ever weakened.

Also corrected an assertion in the previous commit's test, which counted
any non-None result as an escape. '....', '~' and 'a\..\..' are ordinary
directory names on Linux and resolve safely inside the base; treating
them as escapes made the test fail on correct code.

35 tests. The 5 test_web_api.py failures are pre-existing on this branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-21 18:25:50 -04:00
ChuckBuildsandClaude Opus 5 79ba93f5a6 fix(composer): recognisable path containment, and define md:inline
Follow-up to the previous commit, which made the CodeQL count worse
rather than better: 19 alerts became 22. Two mistakes.

First, the containment check used `base not in candidate.parents`.
That is correct Python but not a form static analysis recognises, so
every path-injection alert stayed and _plugin_dir itself picked up two
more. It now uses os.path.realpath plus os.path.commonpath, which is
both the documented sanitiser shape and stricter than the obvious
alternative: "/x/plugins-evil" startswith "/x/plugins" but is a
different directory, and there is now a test that fails if anyone
swaps commonpath for startswith.

Second, raising ComposerInputError from _plugin_dir and returning
str(exc) added two new py/stack-trace-exposure alerts -- CodeQL flags
exception text reaching a response regardless of the exception's type.
_plugin_dir returns None instead and the three handlers answer with a
fixed literal. There is nothing a caller needs there beyond "that id is
not ok".

Also defines .md\:inline in app.css. composer.html marks five toolbar
button labels `hidden md:inline`, and the class was never defined, so
those labels were hidden at every width and the buttons stayed
icon-only. main's test_web_static_audit.py catches it -- the branch
predates that test, which is why it only surfaced now that CI checks
the merge:

    Responsive utility classes referenced in templates but never
    defined in app.css (they silently no-op): ['md:inline']

Verified against the merged state -- main's app.css plus this one line,
audited against this branch's templates: 3 passed. The other twelve
classes the audit flags locally are defined on main and are artifacts of
this branch being 54 commits behind.

33 containment tests. Mutation-checked twice: removing the containment
lets eight payloads escape, including /etc/passwd and
plugin/../../../../../../etc/shadow; swapping commonpath for startswith
fails the sibling-prefix test.

Not addressed: three py/stack-trace-exposure alerts on the
_generate_plugin_files handlers. Those return str(exc) for
ComposerInputError, whose seven raise sites are all authored literals
("Author is required.", "Config variable key X is not a valid Python
identifier."). Suppressing them means replacing useful validation
feedback with a generic string, which is a real cost to the user for a
scanner's benefit. Worth a decision rather than a silent downgrade.

The 5 test_web_api.py failures are pre-existing on this branch --
identical counts with these changes stashed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-21 17:52:11 -04:00
ChuckBuildsandClaude Opus 5 e499efb1f0 fix(composer): resolve plugin paths at the filesystem boundary
CodeQL reports 19 alerts against this PR -- 16 high-severity
py/path-injection plus 3 py/stack-trace-exposure -- all in
web_interface/blueprints/composer.py, where a request-supplied plugin_id
reaches Path(plugins_dir) / plugin_id and the result is created, written
to, deleted with shutil.rmtree, and read back.

The path-injection alerts are false positives today. _PLUGIN_ID_RE is
fully anchored and permits only [a-z][a-z0-9-]{0,62}, so every traversal
payload is already rejected; I checked fourteen of them, including
../../etc/passwd, a/../../etc, /etc/passwd and encoded variants, and none
gets past it.

They are worth fixing anyway. The guarantee lived in a regex several
hundred lines from the path building, so relaxing that pattern later --
to allow an underscore, say -- would open a traversal with nothing at the
filesystem boundary to catch it. _plugin_dir() now resolves the candidate
and refuses anything that is not inside plugins_dir, and all three call
sites go through it. That is also the shape static analysis recognises,
which is why sixteen alerts landed on code that was already safe.

The regex anchor moves from $ to \Z. Python's $ also matches just before
a trailing newline, so "myplugin\n" was accepted and would have created a
directory whose name ends in one. Not traversal, but not a name anything
downstream should have to handle.

For the stack-trace exposure: the handlers returned str(exc) for any
ValueError out of _generate_plugin_files. The seven raises there are all
curated, user-facing validation messages, and they now use a
ComposerInputError subclass so they keep reaching the user verbatim. A
ValueError from anywhere else -- json, int(), a library -- is logged with
a traceback and answered generically, since its text can name internal
paths.

Verified: 32 tests covering fourteen traversal payloads and twelve
malformed ids. The key one re-runs every payload with the id pattern
deliberately loosened to allow slashes and dots; removing the containment
check fails it with

    these escaped the base with a loosened regex:
      [('/etc/passwd', '/etc/passwd'), ('//etc/passwd', '//etc/passwd')]

so the boundary is doing real work rather than shadowing the regex.

The 5 failures in test_web_api.py are unrelated and pre-existing on this
branch -- identical counts with these changes stashed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-21 17:18:26 -04:00
ChuckBuildsandClaude Sonnet 5 cd7e16e58e fix(security): validate plugin_id before path construction in /api/install
CodeQL flagged 16 high-severity "path depends on user-provided value"
alerts. Investigated each:

- install_locally() (/api/install) built a filesystem path from
  metadata.id without validating it at that point -- it was only
  implicitly safe because _generate_plugin_files() validates the same
  field (re-extracted independently) earlier in the same request. That's
  a real gap: reorder or change that earlier call and it's an exploitable
  path traversal / arbitrary file write. Fixed by validating plugin_id
  directly against _PLUGIN_ID_RE at the point the path is built, matching
  the pattern already used correctly in validate_id() and load_plugin().
- The other 10 flagged locations (serve_font's allowlist check,
  validate_id, load_plugin and its downstream reads) were already
  guarded by an explicit check earlier in the same function -- false
  positives from CodeQL not modeling those as sanitizers.

Also fixed 2 of the 5 "stack trace exposed" warnings that were genuine:
install_locally() and load_plugin() returned raw OSError/Exception text
to the client in a 500 response; now logged server-side with a generic
client-facing message. The other 3 (generate_zip/install_locally/
preview_code returning str(ValueError) from _generate_plugin_files) are
deliberate, human-authored validation messages, not exception internals
-- left as-is.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-14 17:23:25 -04:00
ChuckBuildsandClaude Sonnet 5 47e3021fc3 fix: address Codacy findings in the Composer blueprint
- Dropped a pointless f-string prefix (no placeholders) on the default
  plugin description.
- Replaced two bare except:pass/continue blocks (manifest.json listing,
  config_schema.json parsing) with a logged warning before falling
  through to the same skip-this-entry behavior -- same control flow,
  now visible in logs instead of silent.

Skipped as false positives (verified against actual usage, not fixed):
- Jinja2 Environment(autoescape=False) -- this env renders manager.py.j2,
  a Python source-code generator, never HTML; autoescaping would corrupt
  generated code. Flagged by a generic XSS rule that assumes all Jinja2
  environments render HTML.
- "Flask route directly returning a formatted string" on _as_rgb_filter
  -- that's a Jinja *filter* function, not a Flask route.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-14 16:31:43 -04:00
ChuckBuildsandClaude Sonnet 5 e319540c6e feat(web): add Plugin Composer -- visual drag-and-drop plugin builder
Web UI (/composer/) for building a working LEDMatrix plugin without
writing Python: drop elements (text, time, date, countdown, scrolling
text, bar/waveform, groups, custom config variables) onto a canvas
matching the real panel's pixel grid, configure them with live preview,
then generate a real plugin (manager.py + manifest.json + config_schema.json)
from manager.py.j2 -- downloadable as a ZIP or installed directly.

NOTE: composer_bp is not yet registered in web_interface/app.py, so this
blueprint is currently inert. Split out of the original chore/dead-code-
removal commit, which had accidentally bundled this in alongside unrelated
dead-code deletions; app.py registration was not part of that commit
either and still needs to be added before this is reachable.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-14 16:27:50 -04:00
56 changed files with 6369 additions and 6681 deletions
-75
View File
@@ -17,81 +17,6 @@ release that ships it.
accepts both, but the store flags the old spelling as deprecated
(`store_manager.py`) and only the new one is in `schema/manifest_schema.json`.
## 3.3.0
**The release the sports scoreboards floor on to delete their bundled copies.**
3.2.0 shipped the unified sports library and made `ledmatrix_min_version`
enforceable; this ships the last three shared modules and completes the store
gate, so a scoreboard can now floor here and carry no fallback at all.
New modules a plugin may import via `src.*` and floor on 3.3.0 for:
- `src/common/sports_card.py` — settings, colour, font and date helpers for a
scoreboard's `game_renderer.py`. Free functions taking `config`/`fonts`
explicitly, so nothing about the caller's class is assumed.
- `src/common/sports_game_renderer.py` — `SportsGameRendererMixin`: scroll/Vegas
card geometry (centre gap, logo slot and cache key, layout offsets, the
upcoming-card date and time layout). No `__init__` and no state, so adoption
is one line on the class statement.
- `src/common/sports_shared.py` — `SportsCoreSharedMixin`,
`SportsLiveSharedMixin`, `SportsRecentSharedMixin`: the `sports.py` bodies
byte-identical in all eight lineage-sharing scoreboards.
All three sit under `src/common/` rather than `src/base_classes/sports/`,
deliberately: importing that package pulls `core.py` → `DisplayManager` →
`rgbmatrix`, and these are pure logic. Plugins importing them must not acquire a
hardware dependency.
Three notes for anyone adopting `sports_shared`:
- `SportsRecentSharedMixin` defines `__init__`. Its bare `super()` binds to the
mixin, so it reaches the host only when the mixin is listed **first** in the
bases. Reversing that order silently skips the host constructor.
- Methods that resolve the plugin's `config_schema.json` use `_plugin_dir()`,
which walks the MRO rather than reading `__file__` — `__file__` is now
`src/common/`. It walks because `SportsCore` is an ABC: a subclass built with
`type(name, bases, ns)` reports `__module__` as `"abc"`.
- `_get_timezone`, `_extract_game_details` and `_fetch_data` are byte-identical
across the eight but stay in the plugins. The first binds a per-plugin
timezone module whose contents differ; the other two are the abstract stubs
that define the sport.
**The store's compatibility gate is now on every registry-managed route.** 3.2.0
gated `install_plugin`. This release gates the git-pull update path and
`install_from_url`, so a plugin whose floor the core cannot meet is refused
after download with no partial directory left behind.
### Fixed
- **ESPN 403s.** `site.api` began rejecting the User-Agent strings this repo
sent on 2026-08-04; every shared-data-source scoreboard returned
`403 Forbidden`. Requests now send an identifying token with a project URL —
browser-style strings and bare custom tokens are both refused.
- **Low-memory boards becoming unreachable under load** while still answering
pings and serving the web UI. Fetched payloads are released after delivery
rather than pinned on the completed request for up to an hour, malloc arenas
are capped, and log volume and SD writes are reduced. Available memory is now
reported in Tools diagnostics.
- **A failed logo download pinning a team to a grey box**: the placeholder was
written under the real logo's filename, so later attempts found a file and
reported success without retrying.
- **A plugin enabled but never loaded is retried** rather than staying absent
with `error = null`.
- `ttl` now controls cache expiry; abandoned cache writes no longer leave temp
files; one cache-cleanup thread per directory rather than per manager.
- Odds are fetched for the games displayed, not the whole schedule window, and a
stalled ESPN no longer stalls the whole plugin update.
- Array-item secrets are no longer wiped or logged.
- A restored backup matches the device it was taken from.
- The web UI reports the real error instead of "unknown", rejects non-finite
JSON numbers, and stops checkbox groups posting back hidden options.
### Added
- `display.hardware.orientation` for panels mounted upside down.
- Vegas keeps live content in the ticker rather than being preempted by it.
- Schemas can label enum dropdown options.
## 3.2.0
**The first release shipping the unified sports library.** This is the version
+2 -4
View File
@@ -161,11 +161,9 @@ The system supports live, recent, and upcoming game information for multiple spo
### LED Matrix Panels
(2x in a horizontal chain is recommended)
- [Adafruit 64×32](https://www.adafruit.com/product/2278) – designed for 128×32 but works with dynamic scaling on many displays (pixel pitch is user preference)
**Warning: Lately the Waveshare Panels have had different variations - only some are compatible with this project. I hope to identify what is different to fix it but so far there is a decent chance you get a mis-matched set of panels if you don't buy them all at once! **
- [Waveshare 64×32](https://amzn.to/3Kw55jK) - Does not require E addressable pad
- [Waveshare 96×48](https://amzn.to/4bydNcv) – higher resolution, requires soldering the **E addressable pad** on the [Adafruit RGB Bonnet](https://www.adafruit.com/product/3211) to “8” **OR** toggling the DIP switch on the Adafruit Triple LED Matrix Bonnet *(no soldering required!)*
- There are some Panels on Aliexpress that have worked fine for me, shop around! I think Adafruit is probably the "safest" but they do have some limitation on resolution and layout.
> Amazon Affiliate Links – ChuckBuilds receives a small commission on purchases
- [Waveshare 96×48](https://amzn.to/4bydNcv) – higher resolution, requires soldering the **E addressable pad** on the [Adafruit RGB Bonnet](https://www.adafruit.com/product/3211) to “8” **OR** toggling the DIP switch on the Adafruit Triple LED Matrix Bonnet *(no soldering required!)*
> Amazon Affiliate Link – ChuckBuilds receives a small commission on purchases
### Power Supply
- [5V 4A DC Power Supply](https://www.adafruit.com/product/658) (good for 2 -3 displays, depending on brightness and pixel density, you'll need higher amperage for more)
Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 105 KiB

+37 -169
View File
@@ -27,7 +27,7 @@ These are independent concerns. Conflating them is what produces god classes.
| Plugin loads on a core that predates a module | Guarded import with a bundled fallback (`try: from src.X import Y / except ModuleNotFoundError: from y import Y`) |
| Plugin loads on a core that predates a *method* | Capability probing — `hasattr(SportsCore, "_detect_stale_games")` — never a version comparison. The loader's compat check is advisory-only (it logs and continues), so probing is the real protection. |
| Core changes never break a plugin's rendering | The **view-model contract**: `_extract_game_details_common` returns a dict whose `GUARANTEED_KEYS` are frozen by `test/test_skin_system.py::TestViewModelContract`. Keys may be added, never renamed or removed. |
| A plugin can drop its bundled copy safely | The **sunset rule**: its manifest must floor `ledmatrix_min_version` at the first core release shipping the module (recorded in `CHANGELOG.md`) — *necessary but not sufficient*. The store enforces that floor on every registry-managed install and on both supported update paths (sideloading via `install_from_url` is not gated), but a floor cannot reach a user who never updates, so the copy also waits for the B6 gate below. |
| A plugin can drop its bundled copy safely | The **sunset rule**: its manifest must floor `ledmatrix_min_version` at the first core release shipping the module (recorded in `CHANGELOG.md`) — *necessary but not sufficient*. Nothing enforces that floor today, so the copy also waits for the B6 gate below. |
The core API is **additive-only**. A method the plugins call is never removed or
given a new required parameter; new behavior arrives as new methods with
@@ -204,7 +204,7 @@ legacy compatibility rather than the mechanism.
B0–B3 are merged and shipping in core 3.2.0. Everything that remains is
**rollout**, and it splits into three phases with very different risk profiles.
The original plan folded the last two together; they are separated here because
one of them cannot break a user on an old core and the other can.
one of them is safe by construction and the other is not.
| Phase | Scope | Status | Gate |
|---|---|---|---|
@@ -212,9 +212,9 @@ one of them cannot break a user on an old core and the other can.
| **B1** | Promote the nine universal methods; convert `sports.py` → package | ✅ | Characterization suite green; no behavior change intended |
| **B2** | `CelebrationMixin` + rotation strategies as opt-in capabilities | ✅ | Non-adopters have zero new code in their MRO; strategies checked against verbatim plugin transcriptions |
| **B3** | Upstream the scroll **orchestration** layer as `src/common/sports_scroll.py`, reading `global_config['target_fps']` natively | ✅ | Content building stays per-sport |
| **B4** | Ship 3.2.0 *and* make version reporting trustworthy | ✅ | Released 2026-08-03; tag, release and `src.__version__` agree; compatibility gate merged (#428, #431, #433) |
| **B5** | Adoption — guarded core imports, all eight. **Bundled copies stay.** | ✅ | All eight adopted; harness byte-identical; see the B5 retrospective below — four shipped broken and were repaired in plugins #251 |
| **B6** | Sunset — delete the bundled copies | ✅ | Ran 2026-09-01, all eight. Floors at 3.2.0; the store refuses on all three routes in (#431/#433, #508, #510). See "B6 — what actually happened" |
| **B4** | Ship 3.2.0 *and* make version reporting trustworthy | ⏳ **next** | Tag, release, and `src.__version__` agree; compatibility gate merged |
| **B5** | Adoption — guarded core imports: three pilots, then the remaining six. **Bundled copies stay.** | after B4 | Per plugin: harness + goldens byte-identical, then a device soak |
| **B6** | Sunset — delete the bundled copies | **blocked** | B4's gate shipped *and* in users' hands (see below) |
### B4 — what "ship 3.2.0" actually requires
@@ -234,20 +234,6 @@ a floor can be trusted against, and today it is not:
compares the plugin's manifest version against the registry's
`latest_version` and nothing else.
*Fixed, in two parts.* `install_plugin` gained the gate in #431/#433, which
covers every registry-managed install and, through `_reinstall_with_rollback`,
the update path that re-downloads.
`update_plugin`'s git branch pulls in place and re-downloads nothing, so it
stayed ungated until `_gate_pulled_commit` closed it — checked after the pull
(the registry carries no floor field, so the incoming floor is unknowable
before it) and undone with `git reset --hard` to the pre-pull commit. That
route is rare in practice, since monorepo plugins install as archives; it was
closed because the sunset rule in the plugins repo's
`08-shared-sports-code.md` states as **condition 3** that the core enforces
the floor "at install/update time", and B6 rests on that being true rather
than merely written down. `install_from_url` — sideloading a plugin from a
URL — is still ungated.
So B4 is: tag and release 3.2.0; make the tag, the release, and `__version__`
agree, and keep them agreeing; reconsider the `< 2.0.0` skip; migrate manifests
from `ledmatrix_min` to `ledmatrix_min_version`; and add the install/update
@@ -276,21 +262,13 @@ default is the more restrictive.
`compatible_versions`. No manifest still carries it, so there is nothing to
migrate there.)
### B5 — the *fallback* is safe by construction; the modern path is not
The heading matters, because the unqualified version of this claim is false and
this document proves it two sections down: four of the eight adopted plugins
shipped with scroll mode broken on a 3.2.0 core. What is safe by construction is
narrower than "adoption".
### B5 — adoption is safe by construction
A plugin adopting core imports keeps its bundled copy and reaches it through the
guarded import (see the Upgradability table above). On a core that doesn't ship
the module the plugin falls back and behaves exactly as it does today. That
fallback compatibility — and only that — is safe by construction. On a core that
*does* ship the module, correctness is not automatic — object-level and scroll-mode validation
(building both classes and comparing, per the retrospective below) is required
to prove full behavior. There is no version of this step that breaks a user *on
an old core*, which is why it does not wait for B6's gate.
guarded import (see the Upgradability table above). On a core that ships the
module the plugin uses core code; on one that doesn't it falls back and behaves
exactly as it does today. There is no version of this step that breaks a user,
which is why it does not wait for B6's gate.
The hockey scroll-display pilot is **already validated**: adopted against a core
carrying 3.2.0, `scroll_display.py` went from 691 to 289 lines and all 16 harness
@@ -319,8 +297,7 @@ been in users' hands long enough that the population running a core without it
is small.** The bundled copies cost disk space; deleting them early costs
scoreboards, silently. That trade is not close.
Before the first sunset, add a **compatibility regression test**. **Built:**
core `test/test_sports_sunset_matrix.py` (#505). It has to
Before the first sunset, add a **compatibility regression test**. It has to
cover four cases, not one — B5's safety claim and B6's failure mode are
different propositions and only the second is obvious:
@@ -344,144 +321,35 @@ gate rather than trusting the failure to be noticed.
The same suite should exercise the install/update gate, since it is the other
half of the guarantee.
### B6 — what actually happened
**Ran 2026-09-01, across all eight scoreboards.** Held from 2026-08-05 to
2026-09-01 on the argument below, which is kept because the reasoning applies to
the next module, not because it is still in force.
**The hold, and why it lifted.** The stated gate was evidence of 3.2.0 uptake —
"a few months of it being the default download, or store-side install data".
That evidence never arrived and could not: the core updates by
`git pull --rebase`, so release-asset counts cannot measure uptake, and no
store-side telemetry exists. What changed instead is that the *risk* the gate
protected against was closed directly. The store now refuses a plugin whose
floor exceeds the running core on **all three** routes in:
| route | gated by |
|---|---|
| `install_plugin` — every path that re-downloads, `_reinstall_with_rollback` included | #431, #433 |
| `update_plugin`'s git branch — pulls in place, re-downloads nothing | #508 |
| `install_from_url` — sideloading | #510 |
With all three closed a pre-3.2.0 user cannot receive a sunset plugin at all;
they keep the version they already run. The population the hold existed to
protect is protected by refusal rather than by a bundled copy — which is what
the copy was standing in for.
**What shipped.** Eight plugins, ~5,800 lines of frozen fallback deleted. Each:
copy removed, guarded import collapsed to a plain one, floor raised to 3.2.0,
`test_core_fallback.py` rewritten as `test_core_scroll.py` asserting the sunset
rather than the fallback. `scripts/check_scroll_adoption.py` gained
`sunset_violations` and a `SUNSET_PLUGINS` set naming all eight, so a
resurrected copy or a returned guard fails CI.
Delivered as plugins #346 (hockey, later folded into #351), #349 (football),
#350 (baseball), #351 (the remaining six).
**Two things found by doing it, both worth carrying forward:**
- **Only one fallback held orchestration logic the core lacked.** baseball's
`_configure_scroll_helper` reinterpreted `scroll_speed` as pixels-per-*frame*
when `speed × delay` fell outside the 0.1–5.0 window — measured, 10–20×
faster than configured for a speed between 1.0 and 5.0. Standardised onto the
core's behaviour (honour the documented px/sec, clamp) rather than preserved.
Every other difference across the eight was a docstring, an unreachable
`scroll_helper is None` guard, or an equivalent diagnostic.
- **Two tests had been leaning on the guard without anyone knowing.**
`soccer/test_live_screens.py` stubbed `src` in a way that shadowed the core,
so its guarded import fell back and the test had been exercising the *frozen
copy* rather than the shipping class since B5. Before sunsetting anything else
that carries a guarded core import, grep for tests that stub `src`.
**The floor-raising traps still apply** to any future sunset: four plugins
declare their floor top-level, where editing `versions[0]` is a silent no-op,
and the floor has three live spellings (`min_ledmatrix_version`,
`requires.min_ledmatrix_version`, `versions[].ledmatrix_min_version`, plus
deprecated `ledmatrix_min`). See
`src/plugin_system/compatibility.py:declared_min_version` for the resolution
order any floor-raising tool must reproduce — and note the name is **inverted**
between the top level and `versions[]`.
**Still not adopted, deliberately:** `data_sources.py`, `game_renderer.py` and
`base_odds_manager.py`. The standing decision held them until B6 closed; it now
has, so they can be reconsidered — with B5's lesson applied, which is to build
the object and diff rendered output rather than trust a static check.
### B5 retrospective — what the adoption actually cost
Recorded because it is the evidence behind the two decisions above, and because
"the adoption went fine" is not what happened.
**Four of the eight shipped with scroll mode broken** on a 3.2.0 core, and were
repaired in plugins-repo #251. The restructure lifted the content methods
verbatim but left the state they read off `self` behind: separator-icon
constants (hockey, basketball, lacrosse) and the game-renderer cache (afl).
hockey/basketball/lacrosse could not construct the scroll display at all; afl
raised inside `prepare_scroll_content`, which the core base *catches*, so its
only symptom was scroll mode silently drawing nothing.
Three things are worth carrying forward:
- **The bundled fallback did not protect anyone from this.** The break was on
the modern path, which the fallback never touches. Carrying the second copy
bought nothing against the actual defect while creating the divergence that
produced it. That is an argument *for* B6, not against it.
- **Every gate was green.** The safety harness renders the scoreboard screens,
not scroll mode; `test_core_fallback.py` checked that methods existed and that
their *globals* resolved, and `self.NHL_SEPARATOR_ICON` is an attribute read,
invisible to an AST scan for `Name` loads. The fix was to stop reasoning about
source and **build the object**: construct both classes on both paths, compare
the separator icons they end up with, and assert the adopted class ends up
with every instance attribute the bundled one sets.
- **Test what the change touches, not what is convenient to render.** Scroll
mode had no coverage because the harness could not reach it. A comparison
harness that renders the same games through both paths and diffs the pixels
needs no per-sport knowledge of the right answer, only that adopting core code
did not change it.
**The ledger.** Before adoption, eight duplicated copies totalled 5,685 lines.
After adoption plus the frozen legacy copies it was 10,610; removing the dead
inline duplication (plugins #252) brought it to roughly 8,620. B6 would take it
to about 3,300 including the shared core module — some 2,400 fewer than before
this project started. **Until B6 runs, the adoption is net negative on disk**,
and its one delivered user-visible gain is that adopted plugins honour the
global `target_fps` instead of hardcoding ~100 FPS.
### Decision: stop adopting further modules until B6 closes
`data_sources.py` (9 copies), `game_renderer.py` (8) and `base_odds_manager.py`
are the obvious next candidates. **Do not adopt them yet.** Each adoption adds
carrying cost — a second copy to keep in step — against a payoff that is
contingent on B6, and B6 is gated on an installed base we cannot currently
measure. Consolidate what is already committed; revisit when B6 does.
## What's next
Steps 1–5 of the original plan are **done**: 3.2.0 is tagged and published with
a version number CI now asserts (#428), the compatibility gate is in
`install_plugin` and reads `compatible_versions` as well as the floor
(#431, #433), the newest manifest entry is required to use `ledmatrix_min_version`
(plugins #244), and all eight plugins have adopted the scroll orchestration
(plugins #245–#249, repaired in #251, tidied in #252).
In order. Each step is independently useful and independently revertible.
What actually remains, smallest first:
1. **Soak the adoptions on hardware.** football and hockey have been run on a
live rig through real games; baseball was watched through one earlier. The
rest are proven by harness, unit tests and pixel comparison. Out-of-season
sports cannot be soaked until their season starts. When you do, **check the
rig's `*_display_mode` first** — a board in `switch` mode will happily load a
sunset plugin and tell you nothing about the scroll code the sunset changed.
2. **Cut 3.3.0.** Not required by B6 — its floors are 3.2.0, which is released —
but `calendar` 1.2.3 floors at 3.3.0 for the device-authorization endpoints
that landed after 3.2.0, so it is un-installable until the release exists.
3. **Reconsider the held modules** (`data_sources.py`, `game_renderer.py`,
`base_odds_manager.py`) now that the sunset has closed. `game_renderer.py` is
the largest single duplication left: ~11,500 lines across eight plugins, with
~36,500 more in the eight `sports.py`. Note that core already ships
`src/base_classes/sports/` (~143KB, promoted in B1/B2) that **no plugin
imports** — check whether it has drifted before treating it as the target.
1. **Tag and publish v3.2.0.** The code is already on `main` (`21825cbf`).
Nothing else blocks this, and it is what makes `ledmatrix_min_version:
"3.2.0"` refer to something real.
2. **Make the version number honest.** Have the release process assert that the
tag, the GitHub release, and `src.__version__` agree — a check in CI is
cheaper than the confusion of the last two releases. Then revisit the
`< 2.0.0` skip in `_warn_if_incompatible`, which currently silences the
warning for the users who most need it.
3. **Add the compatibility gate** to `StoreManager.install_plugin` and
`.update_plugin`: refuse a plugin whose declared floor exceeds
`src.__version__`, and surface the reason in the store UI rather than only
the log. This is the single change that turns the floor from documentation
into a guarantee, and B6 depends on it.
4. **Migrate the manifests** to `ledmatrix_min_version`, and reconcile them with
`compatible_versions` (see above — that field is the required, canonical one,
and the gate does not read it yet). Currently 28 plugins spell the floor both
ways across their `versions[]` entries, 12 use only the old spelling, and 2
only the new. Scope the sweep to the nine sports plugins if a 42-plugin
version-bump wave isn't worth it — but the `compatible_versions` half has to
cover every manifest the gate can refuse, or define explicit legacy handling,
before the gate is allowed to block anything.
5. **Run B5 adoption** — hockey, soccer, football, then the remaining six.
Bundled copies stay. Byte-identical harness output per plugin, then a soak.
6. **Only then plan B6**, with the compatibility regression test described above
in CI first.
## How to keep this project healthy
+4 -7
View File
@@ -1740,15 +1740,12 @@ journald_effective() {
systemd-analyze cat-config systemd/journald.conf >/dev/null 2>&1; then
systemd-analyze cat-config systemd/journald.conf 2>/dev/null
else
cat /etc/systemd/journald.conf /etc/systemd/journald.conf.d/*.conf 2>/dev/null || true
cat /etc/systemd/journald.conf /etc/systemd/journald.conf.d/*.conf 2>/dev/null
fi
}
journald_conf="$(journald_effective)"
# Both settings are optional, and stock images ship them commented out. grep
# exits 1 on no match, which pipefail turns fatal under set -e — an absent
# setting must read as empty, not abort the install.
journald_storage="$(printf '%s\n' "$journald_conf" | grep -E '^[[:space:]]*Storage=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]' || true)"
journald_cap="$(printf '%s\n' "$journald_conf" | grep -E '^[[:space:]]*SystemMaxUse=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]' || true)"
journald_storage="$(printf '%s\n' "$journald_conf" | grep -E '^[[:space:]]*Storage=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]')"
journald_cap="$(printf '%s\n' "$journald_conf" | grep -E '^[[:space:]]*SystemMaxUse=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]')"
if [ "$journald_storage" = "persistent" ] && [ -n "$journald_cap" ]; then
echo "Persistent journald storage already configured (SystemMaxUse=$journald_cap)"
@@ -1774,7 +1771,7 @@ else
# after ledmatrix-persistent.conf (zz-local.conf and friends) still wins.
# Writing the file is not evidence it took effect -- re-read and say so
# plainly rather than reporting success we cannot confirm.
journald_now="$(journald_effective | grep -E '^[[:space:]]*Storage=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]' || true)"
journald_now="$(journald_effective | grep -E '^[[:space:]]*Storage=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]')"
if [ "$journald_now" = "persistent" ]; then
echo " Persistent journald storage active"
else
+1 -1
View File
@@ -4,5 +4,5 @@ LEDMatrix Display System
Core source package for the LED Matrix Display project.
"""
__version__ = "3.3.0"
__version__ = "3.2.0"
+11 -210
View File
@@ -14,12 +14,11 @@ Key Features:
- Memory-efficient data storage
"""
import itertools
import time
import logging
import threading
import requests
from typing import Dict, Any, Optional, Callable, List
from typing import Dict, Any, Optional, Callable
from dataclasses import dataclass, field
from enum import Enum
import queue
@@ -51,33 +50,14 @@ class FetchRequest:
max_retries: int = 3
priority: int = 1 # Higher number = higher priority
callback: Optional[Callable] = None
# Callbacks from submitters that JOINED this fetch instead of starting a
# duplicate one. The primary `callback` above belongs to whoever created
# the request; these belong to everyone who asked for the same cache_key
# while it was still in flight.
extra_callbacks: List[Callable] = field(default_factory=list)
created_at: float = field(default_factory=time.time)
status: FetchStatus = FetchStatus.PENDING
# Set once the worker has decided this response will be cached, while it
# still holds the lock. From that point cancelling is refused: the write
# is already authorised, and abandoning it here would put the payload in
# the cache with the callbacks suppressed -- joiners waiting forever for a
# fetch that did, in fact, succeed.
commit_claimed: bool = False
result: Optional[Any] = None
error: Optional[str] = None
@dataclass
class FetchResult:
"""Result of a background fetch operation.
``data`` survives on the stored result only for requests submitted without
a ``callback``, where polling ``get_result()`` is the sole way to collect
it. When a callback was given, the payload has already been delivered and
the service releases it -- see :meth:`BackgroundDataService._release_payload`.
Either way the data remains in the cache under the request's ``cache_key``,
which is where consumers read it from.
"""
"""Result of a background fetch operation."""
request_id: str
success: bool
data: Optional[Any] = None
@@ -86,10 +66,6 @@ class FetchResult:
fetch_time: float = 0.0
retry_count: int = 0
completed_at: float = field(default_factory=time.time) # Timestamp when request completed
# The request's final status, recorded so a finished request can still be
# reported accurately. Without it a caller can only be told COMPLETED or
# FAILED, which turns "you cancelled this" into "this errored".
final_status: Optional[FetchStatus] = None
class BackgroundDataService:
"""
@@ -114,20 +90,6 @@ class BackgroundDataService:
# Thread management
self.executor = ThreadPoolExecutor(max_workers=max_workers, thread_name_prefix="BackgroundData")
# cache_key -> request_id for fetches currently in flight. Submitting
# the same key twice used to start two identical fetches: request_id
# carries a millisecond timestamp, so every submit looked new, and
# active_requests is keyed by it rather than by what is being fetched.
# On a real board the season-schedule key is requested by both the
# Recent and the Upcoming manager, which miss the cache in the same
# millisecond and each download and parse the same payload.
self._inflight_by_cache_key: Dict[str, str] = {}
# request_id was sport_year_milliseconds, which is not unique: two
# submits inside the same millisecond produced the SAME id, so one
# silently replaced the other in active_requests and completed_requests.
# Rare before, but dedupe hands this id back to every joiner as their
# handle for get_result(), so it has to be unique. A counter is enough.
self._request_seq = itertools.count()
self.active_requests: Dict[str, FetchRequest] = {}
self.completed_requests: Dict[str, FetchResult] = {}
self.request_queue = queue.PriorityQueue()
@@ -215,9 +177,7 @@ class BackgroundDataService:
if cache_key is None:
cache_key = self.get_sport_cache_key(sport)
with self._lock:
request_id = (f"{sport}_{year}_{int(time.time() * 1000)}"
f"_{next(self._request_seq)}")
request_id = f"{sport}_{year}_{int(time.time() * 1000)}"
# Check cache first
cached_data = self.cache_manager.get(cache_key)
@@ -231,19 +191,14 @@ class BackgroundDataService:
cached=True,
fetch_time=0.0
)
# Filed before the callback runs, as it always was: a callback
# that queries get_result()/is_request_complete() for its own
# request must still find it. Releasing afterwards mutates the
# same object the dict holds.
self.completed_requests[request_id] = result
if callback:
try:
callback(result)
except Exception as e:
logger.error(f"Error in callback for request {request_id}: {e}")
self._release_payload(result)
logger.debug(f"Cache hit for {sport} {year} data")
return request_id
@@ -263,29 +218,7 @@ class BackgroundDataService:
)
with self._lock:
existing_id = self._inflight_by_cache_key.get(cache_key)
existing = self.active_requests.get(existing_id) if existing_id else None
if existing_id and existing is None:
# Stranded index entry: the request it names is gone. Drop it and
# fetch normally. Looking the request up rather than trusting the
# id is what stops a stale entry wedging a key forever.
del self._inflight_by_cache_key[cache_key]
if existing is not None:
# Someone is already fetching this key. Ride along rather than
# duplicating the download, the parse and the resident copy.
if callback:
existing.extra_callbacks.append(callback)
self.stats['deduplicated_requests'] = (
self.stats.get('deduplicated_requests', 0) + 1
)
logger.info(
"Joined in-flight fetch %s for %s (cache_key=%s) instead of "
"starting a duplicate", existing_id, sport, cache_key
)
return existing_id
self.active_requests[request_id] = request
self._inflight_by_cache_key[cache_key] = request_id
self.stats['total_requests'] += 1
self.stats['cache_misses'] += 1
@@ -310,31 +243,7 @@ class BackgroundDataService:
try:
with self._lock:
# A request cancelled while it sat in the executor queue must
# stay cancelled. Overwriting the status here undid the cancel
# outright: the worker went on to download, cache and call back
# for work the caller had already withdrawn.
if request.status == FetchStatus.CANCELLED:
cancelled_before_start = True
else:
cancelled_before_start = False
request.status = FetchStatus.IN_PROGRESS
if cancelled_before_start:
logger.info(
"Request %s was cancelled before its worker started; "
"skipping the fetch entirely", request.id
)
# Assign before returning: the finally block stores `result`
# in completed_requests, so building a fresh one here would
# file the untouched placeholder instead of this outcome.
result = FetchResult(
request_id=request.id,
success=False,
error="cancelled",
fetch_time=time.time() - start_time,
retry_count=request.retry_count
)
return result
request.status = FetchStatus.IN_PROGRESS
logger.info(f"Starting background fetch for {request.sport} {request.year}")
@@ -360,37 +269,6 @@ class BackgroundDataService:
# Log data validation
logger.debug(f"Validated {len(events)} events for {request.sport} {request.year}")
# A cancelled request must not commit anything. Cancelling
# releases the cache_key, so a replacement fetch for the same key
# may already be in flight or finished -- writing this response to
# the cache now would overwrite fresher data with the response
# nobody wanted. The worker has no way to abort the HTTP call, so
# this is where the work gets discarded.
with self._lock:
cancelled = request.status == FetchStatus.CANCELLED
if not cancelled:
# Claim the commit in the same critical section that read
# the status, so a cancel cannot slip in between the check
# and the cache write below. The write itself stays outside
# the lock: it serialises a multi-megabyte payload to the
# SD card, and holding the service lock across that would
# stall every submit, status query and cancel behind it.
request.commit_claimed = True
if cancelled:
logger.info(
"Discarding response for cancelled request %s; %s may "
"already belong to a replacement fetch",
request.id, request.cache_key
)
result = FetchResult(
request_id=request.id,
success=False,
error="cancelled",
fetch_time=time.time() - start_time,
retry_count=request.retry_count
)
return result
# Cache the data
self.cache_manager.set(request.cache_key, data)
@@ -416,12 +294,7 @@ class BackgroundDataService:
logger.error(f"Failed to fetch {request.sport} {request.year} data: {error_msg}")
with self._lock:
# Don't relabel a cancelled request. The callback gate in the
# finally block only suppresses CANCELLED, so promoting it to
# FAILED here delivered an error callback for a fetch nobody
# was waiting on any more.
if request.status != FetchStatus.CANCELLED:
request.status = FetchStatus.FAILED
request.status = FetchStatus.FAILED
request.error = error_msg
result = FetchResult(
@@ -435,26 +308,9 @@ class BackgroundDataService:
finally:
# Store result and clean up
with self._lock:
result.final_status = request.status
self.completed_requests[request.id] = result
if request.id in self.active_requests:
del self.active_requests[request.id]
# Stop accepting joiners and take the callback list in the same
# critical section. A submitter that arrives after this point
# finds no in-flight entry and either hits the cache (written
# above, before the result was built) or starts a fresh fetch --
# what it must never do is join a fetch whose callbacks have
# already run and then never be called.
if self._inflight_by_cache_key.get(request.cache_key) == request.id:
del self._inflight_by_cache_key[request.cache_key]
# A cancelled request delivers nothing: its joiners were told
# about a fetch that has been abandoned, and a replacement will
# call them via its own request.
if request.status == FetchStatus.CANCELLED:
callbacks = []
else:
callbacks = ([request.callback] if request.callback else [])
callbacks.extend(request.extra_callbacks)
# Update statistics
if result.success:
@@ -471,53 +327,14 @@ class BackgroundDataService:
# Periodic cleanup after storing result
self._cleanup_completed_requests()
# Call every callback: the original submitter's and any that joined
# this fetch. One raising must not stop the others being delivered.
for cb in callbacks:
# Call callback if provided
if request.callback:
try:
cb(result)
request.callback(result)
except Exception as e:
logger.error(f"Error in callback for request {request.id}: {e}")
# Released AFTER the loop, not inside it. Every callback here holds
# the same FetchResult, so releasing per-delivery handed the first
# one the data and every joiner `result.data is None` -- which is
# not a quiet degradation: they read `result.data.get('events')` and
# raise AttributeError, which this very loop catches and logs, so
# the symptom was one ERROR line and a manager that silently never
# got its schedule. Deduplication is the normal case, not a corner:
# a sport's recent, upcoming and live managers all ride one season
# fetch.
#
# Guarded on `callbacks`, because a request submitted without one
# has no other way to collect its payload than polling get_result().
# The old per-delivery release got that right by accident: an empty
# list never entered the loop body.
if callbacks:
self._release_payload(result)
request.result = None
return result
@staticmethod
def _release_payload(result: FetchResult) -> None:
"""Drop a delivered payload, keeping the result's status and timings.
Only called once EVERY callback has been handed the data -- callers
that joined an in-flight fetch share this object, so releasing between
deliveries strips the payload out from under the ones still queued.
Consumers read fetched data back from the cache under ``cache_key``;
the copy carried
here was pinning a parsed season schedule -- 946 games for NCAA
football, roughly a tenth of total RAM on a 1GB Pi -- in memory until
the hourly sweep.
The cache-hit path matters most: it runs once per update interval per
sport, mints a fresh request_id each time, and a memory-tier miss
re-parses the payload from disk. Those were genuinely separate copies
accumulating toward the 500-entry cap, not shared references.
"""
result.data = None
def _make_request_with_retry(self, request: FetchRequest) -> requests.Response:
"""
@@ -605,8 +422,6 @@ class BackgroundDataService:
return self.active_requests[request_id].status
elif request_id in self.completed_requests:
result = self.completed_requests[request_id]
if result.final_status is not None:
return result.final_status
return FetchStatus.COMPLETED if result.success else FetchStatus.FAILED
return None
@@ -623,22 +438,8 @@ class BackgroundDataService:
with self._lock:
if request_id in self.active_requests:
request = self.active_requests[request_id]
if request.commit_claimed:
# Too late: the worker holds an authorised commit. Report
# the failure rather than half-cancelling a request whose
# data is about to land in the cache.
logger.debug(
"Not cancelling %s: its response is already being "
"committed", request_id
)
return False
request.status = FetchStatus.CANCELLED
del self.active_requests[request_id]
# Cancelling is the other way a request leaves active_requests,
# so the in-flight index has to be released here too or the key
# stays pointed at a request that no longer exists.
if self._inflight_by_cache_key.get(request.cache_key) == request_id:
del self._inflight_by_cache_key[request.cache_key]
logger.info(f"Cancelled request {request_id}")
return True
return False
+2 -52
View File
@@ -143,10 +143,8 @@ class LogoHelper:
"""
logo_path = Path(logo_path)
# Try to load existing logo first. A placeholder written by a previous
# failed download does not count: it wears the real logo's filename, so
# trusting the file's existence is what left teams as grey boxes.
if logo_path.exists() and not self._is_stale_placeholder(logo_path):
# Try to load existing logo first
if logo_path.exists():
return self.load_logo(team_abbr, logo_path, max_width, max_height)
# Download if URL provided and file doesn't exist
@@ -154,61 +152,13 @@ class LogoHelper:
try:
self.logger.info(f"Downloading logo for {team_abbr} from {logo_url}")
self._download_logo(logo_url, logo_path)
# The file on disk just changed. Any cached image for it is the
# placeholder we came here to replace, and load_logo() answers
# from the cache before touching the disk -- so without this the
# real logo would not appear until the process restarted.
self._invalidate_cached_logo(team_abbr, logo_path)
return self.load_logo(team_abbr, logo_path, max_width, max_height)
except Exception as e:
self.logger.error(f"Failed to download logo for {team_abbr}: {e}")
# The retry failed, so restart the back-off. The stale
# placeholder is still on disk with its old timestamp, and
# leaving it there means the next call retries immediately --
# a download attempt per call, which is what the back-off
# exists to prevent.
self._refresh_stale_placeholder(logo_path)
# Create placeholder if all else fails
return self._create_placeholder_logo(team_abbr, max_width, max_height)
def _invalidate_cached_logo(self, team_abbr: str, logo_path: Path) -> None:
"""Drop every cached size of one logo after its file changed on disk."""
prefix = f"{team_abbr}_{logo_path}_"
for key in [k for k in self._logo_cache if k.startswith(prefix)]:
self._logo_cache.pop(key, None)
if key in self._cache_order:
self._cache_order.remove(key)
@staticmethod
def _refresh_stale_placeholder(logo_path: Path) -> None:
"""Restart the retry back-off after a failed download attempt."""
try:
from src.logo_downloader import refresh_placeholder_timestamp
except ImportError:
return
refresh_placeholder_timestamp(logo_path)
@staticmethod
def _is_stale_placeholder(logo_path: Path) -> bool:
"""True if the file is a placeholder old enough to be worth retrying.
Imported lazily so this module keeps working against a core build whose
logo_downloader predates placeholder marking.
"""
try:
from src.logo_downloader import (
PLACEHOLDER_RETRY_SECONDS,
is_placeholder_logo,
placeholder_age_seconds,
)
except ImportError:
return False
if not is_placeholder_logo(logo_path):
return False
age = placeholder_age_seconds(logo_path)
return age is None or age >= PLACEHOLDER_RETRY_SECONDS
def get_logo_variations(self, team_abbr: str) -> List[str]:
"""
Get possible filename variations for a team abbreviation.
-4
View File
@@ -328,10 +328,6 @@ class ScrollHelper:
elapsed_time = current_time - (self.scroll_start_time or current_time)
# The image already includes display_width padding, so we only need total_scroll_width
required_total_distance = self.total_scroll_width
# Progress telemetry, emitted every few seconds for the whole of
# every scroll. It says how far along a marquee is, which is what
# you turn debug on to watch and not something an operator needs
# in the journal on a device that scrolls all day.
self.logger.debug(
"Scroll progress: elapsed=%.2fs, target=%.2fs, total_scrolled=%.0f/%d px (%.1f%%)",
elapsed_time,
-460
View File
@@ -1,460 +0,0 @@
"""Card-drawing helpers shared by every sports scoreboard plugin.
The eight scoreboards each carried byte-identical copies of the functions
below: the colour pickers, the settings lookup, the date and time formatting,
the favourite-team rules and the font-size grid snapping. One fix had to be
made eight times, and a new scoreboard started by copying them a ninth.
Everything here is a **free function taking explicit arguments**, not a base
class. Adoption is therefore per-function and reversible: a plugin keeps its
method and delegates the body, so the call sites and the override points are
untouched. That is also why `config`, `logger` and `fonts` are parameters
rather than attributes -- the helper never reaches back into the caller.
The bodies are the plugins' own code, moved rather than rewritten. The one
deliberate difference is `crisp_size`, which takes the seven-plugin guard
(`not desired`) instead of football's: they agree on every real input, and
the extra guard only stops a None size raising TypeError.
"""
from datetime import datetime, timezone
from typing import Any, Dict, Optional, Tuple
from zoneinfo import ZoneInfo
__all__ = [
"ELEMENT_FOR_FONT", "FAVORITE_RESULT_COLOR_DEFAULTS", "FONT_NAME_ALIASES",
"FONT_PIXEL_GRID", "MONTH_ABBR", "WEEKDAY_ABBR",
"scroll_card_option", "element_color", "font_color", "coerce_rgb",
"score_color_for", "recent_score_color", "favorite_teams_for",
"side_is_favorite", "side_score", "favorite_result",
"card_tzinfo", "weekday_for", "format_game_date", "format_game_time",
"vs_text", "upcoming_center_mode", "crisp_size", "schema_font_size",
"resolve_font_size", "unshare_element_fonts",
]
#: Which customization element owns each font key, for colour resolution.
ELEMENT_FOR_FONT: Dict[str, str] = {
"score": "score_text",
"time": "period_text",
"team": "team_name",
"status": "status_text",
"detail": "detail_text",
"rank": "rank_text",
}
#: Fallback colours when favourite_result_colors is on but a slot is unset.
FAVORITE_RESULT_COLOR_DEFAULTS: Dict[str, Tuple[int, int, int]] = {
"win": (0, 255, 0),
"loss": (255, 0, 0),
"tie": (255, 200, 0),
}
#: Family aliases the web UI may write, mapped to the shipped filename.
FONT_NAME_ALIASES: Dict[str, str] = {
"press_start": "PressStart2P-Regular.ttf",
"four_by_six": "4x6-font.ttf",
}
#: Pixel grid each face renders crisply on. Off-grid sizes anti-alias, which
#: on an LED matrix is a dim lamp rather than a soft edge.
FONT_PIXEL_GRID: Dict[str, int] = {
"PressStart2P-Regular.ttf": 8,
"4x6-font.ttf": 7,
}
MONTH_ABBR = ("Jan", "Feb", "Mar", "Apr", "May", "Jun",
"Jul", "Aug", "Sep", "Oct", "Nov", "Dec")
WEEKDAY_ABBR = ("Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun")
# ---------------------------------------------------------------------------
# Settings lookup
# ---------------------------------------------------------------------------
def scroll_card_option(config: Optional[Dict[str, Any]], key: str,
default: Any = None) -> Any:
"""Read one key from the scroll_card config block."""
block = (config or {}).get("scroll_card")
if isinstance(block, dict) and block.get(key) is not None:
return block.get(key)
return default
def vs_text(config: Optional[Dict[str, Any]]) -> str:
"""Separator drawn between the teams -- "VS", "@", "at", anything."""
return str(scroll_card_option(config, "vs_text", "VS"))
def upcoming_center_mode(config: Optional[Dict[str, Any]]) -> str:
"""Middle of an upcoming card: 'vs', 'date_time' or 'none'."""
mode = str(scroll_card_option(config, "upcoming_center", "vs") or "vs").lower()
return mode if mode in ("vs", "date_time", "none") else "vs"
# ---------------------------------------------------------------------------
# Colour
# ---------------------------------------------------------------------------
def element_color(config: Optional[Dict[str, Any]], element: str,
default: Tuple[int, int, int] = (255, 255, 255)):
"""Per-element text colour from customization.<element>.text_color."""
try:
cfg = (config or {}).get("customization", {}).get(element, {})
value = cfg.get("text_color")
if isinstance(value, (list, tuple)) and len(value) == 3:
return tuple(max(0, min(255, int(c))) for c in value)
if isinstance(value, str) and value.startswith("#") and len(value) == 7:
return tuple(int(value[i:i + 2], 16) for i in (1, 3, 5))
except (TypeError, ValueError):
pass
return default
def font_color(config: Optional[Dict[str, Any]], fonts: Optional[Dict[str, Any]],
font, default: Tuple[int, int, int] = (255, 255, 255)):
"""Colour for whichever element owns this face.
Matched on identity, and deliberately gives up when one object is
shared: the last-resort font path can hand the same face to several
keys, and there is no right answer for which element's colour that is.
White is what those draws used before, so ambiguity costs nothing.
"""
try:
fonts = fonts or {}
matches = [element for key, element in ELEMENT_FOR_FONT.items()
if fonts.get(key) is font]
if len(matches) == 1:
return element_color(config, matches[0], default)
except (AttributeError, TypeError):
pass
return default
def coerce_rgb(value, fallback):
"""Turn a configured [R, G, B] list into a clamped (r, g, b) tuple."""
# Checked before unpacking: a 3-character string ("123") would otherwise
# iterate into three digits and yield a colour rather than the fallback.
if not isinstance(value, (list, tuple)) or len(value) != 3:
return fallback
try:
r, g, b = (max(0, min(255, int(channel))) for channel in value)
except (TypeError, ValueError):
return fallback
return (r, g, b)
# ---------------------------------------------------------------------------
# Favourite teams
# ---------------------------------------------------------------------------
def favorite_teams_for(config: Dict[str, Any], game: Dict[str, Any]) -> list:
"""Favorite teams that apply to this game.
Both sources are used. Games carry the league manager's *resolved*
favorites, which is the only place dynamic groups such as AP_TOP_25
appear expanded; the config is read as well so an edit takes effect on
already-fetched games, and so hand-built game dicts (tests, other
callers) still work.
"""
favorites = list(game.get("favorite_teams") or [])
league_config = config.get(str(game.get("league", "") or ""))
if isinstance(league_config, dict):
favorites += list(league_config.get("favorite_teams") or [])
else:
favorites += list(config.get("favorite_teams") or [])
return favorites
def side_is_favorite(game: Dict[str, Any], side: str, favorites: set) -> bool:
"""Is the home/away side of this game a favorite team?
Reads both the flat (``home_abbr``) and nested (``home_team.abbrev``)
payload shapes, and matches on the ESPN id too, because a couple of
leagues (NRL) key favorites by id where abbreviations collide.
"""
candidates = [game.get(f"{side}_abbr"), game.get(f"{side}_id")]
team = game.get(f"{side}_team")
if isinstance(team, dict):
candidates += [team.get("abbrev"), team.get("abbreviation"), team.get("id")]
for value in candidates:
if value is not None and str(value).strip().upper() in favorites:
return True
return False
def side_score(game: Dict[str, Any], side: str) -> Optional[int]:
"""Numeric score for one side, from either payload shape."""
raw = None
team = game.get(f"{side}_team")
if isinstance(team, dict) and team.get("score") is not None:
raw = team.get("score")
if raw is None:
raw = game.get(f"{side}_score")
try:
return int(float(str(raw).strip()))
except (TypeError, ValueError):
return None
def favorite_result(config: Dict[str, Any], game: Dict[str, Any]) -> Optional[str]:
"""Say how the favorite team did in a finished game.
Returns 'win', 'loss' or 'tie', or None when there is no single team
to root for: no favorites configured, neither side is a favorite, or
*both* are -- a favorite-vs-favorite game has no losing side worth
flagging in red. Also None when the scores are not usable numbers.
"""
favorites = {
str(team).strip().upper()
for team in favorite_teams_for(config, game)
if str(team).strip()
}
if not favorites:
return None
home_fav = side_is_favorite(game, "home", favorites)
away_fav = side_is_favorite(game, "away", favorites)
if home_fav == away_fav:
return None
home_score = side_score(game, "home")
away_score = side_score(game, "away")
if home_score is None or away_score is None:
return None
if home_score == away_score:
return "tie"
favorite_score, other_score = (
(home_score, away_score) if home_fav else (away_score, home_score)
)
return "win" if favorite_score > other_score else "loss"
def recent_score_color(config: Dict[str, Any], logger, game: Dict[str, Any], default):
"""Fill color for a finished game's score, per favorite_result_colors."""
try:
settings = (config.get("customization") or {}).get(
"favorite_result_colors"
) or {}
if not settings.get("enabled", False):
return default
result = favorite_result(config, game)
if result is None:
return default
return coerce_rgb(
settings.get(f"{result}_color"),
FAVORITE_RESULT_COLOR_DEFAULTS[result],
)
except Exception:
logger.debug("Could not resolve favorite result color", exc_info=True)
return default
def score_color_for(config: Dict[str, Any], logger, game: Dict[str, Any],
game_type: str, default=None):
"""Fill color for a game card's score. Only finished games are tinted.
The default is the configured score colour rather than a flat white,
so customization.score_text.text_color shows on games the favourite
tint does not apply to. The tint still wins where it applies.
"""
if default is None:
default = element_color(config, 'score_text')
if game_type != "recent":
return default
return recent_score_color(config, logger, game, default)
# ---------------------------------------------------------------------------
# Date and time
# ---------------------------------------------------------------------------
def card_tzinfo(config: Optional[Dict[str, Any]], logger):
"""Timezone for weekday/24h conversions; falls back to UTC."""
configured = (config or {}).get("timezone")
if configured:
try:
return ZoneInfo(configured)
except (KeyError, ValueError, TypeError, OSError) as exc:
# KeyError covers ZoneInfoNotFoundError. A bad zone name in
# config should fall back to UTC, not blank the card.
logger.debug("Unusable timezone %r: %s", configured, exc)
return timezone.utc
def weekday_for(config: Optional[Dict[str, Any]], logger,
game: Optional[Dict]) -> str:
"""Weekday abbreviation from the game's start time, or ''."""
if not game:
return ""
raw = game.get("start_time_utc") or game.get("start_time")
if not raw:
return ""
try:
start = raw if isinstance(raw, datetime) else datetime.fromisoformat(
str(raw).replace("Z", "+00:00"))
return WEEKDAY_ABBR[start.astimezone(card_tzinfo(config, logger)).weekday()]
except (ValueError, TypeError):
return ""
def format_game_date(config: Optional[Dict[str, Any]], logger, date_text: str,
game: Optional[Dict] = None) -> str:
"""Format an upcoming card's date per scroll_card.date_format."""
raw = str(date_text or "").strip()
if not raw:
return ""
fmt = str(scroll_card_option(config, "date_format", "abbrev") or "abbrev")
if fmt == "numeric":
return raw
parts = raw.replace("-", "/").split("/")
if not (len(parts) >= 2 and parts[0].strip().isdigit() and parts[1].strip().isdigit()):
return raw
month, day = int(parts[0]), int(parts[1])
if not 1 <= month <= 12:
return raw
name = MONTH_ABBR[month - 1]
if fmt == "numeric_day_first":
return f"{day}/{month}"
if fmt == "day_first":
return f"{day} {name}"
if fmt == "weekday":
weekday = weekday_for(config, logger, game)
return f"{weekday} {name} {day}" if weekday else f"{name} {day}"
return f"{name} {day}"
def format_game_time(config: Optional[Dict[str, Any]], time_text: str) -> str:
"""Return the time as-is (12h) or converted to 24h."""
raw = str(time_text or "").strip()
if not raw or str(scroll_card_option(config, "time_format", "12h")) != "24h":
return raw
cleaned = raw.upper().replace(" ", "")
meridiem = "AM" if cleaned.endswith("AM") else "PM" if cleaned.endswith("PM") else ""
if not meridiem:
return raw
try:
hh, _, mm = cleaned[:-2].partition(":")
hour, minute = int(hh), int(mm or 0)
except ValueError:
return raw
if not (0 <= hour <= 12 and 0 <= minute <= 59):
return raw
hour = hour % 12 + (12 if meridiem == "PM" else 0)
return f"{hour:02d}:{minute:02d}"
# ---------------------------------------------------------------------------
# Font sizing
# ---------------------------------------------------------------------------
#: Per-schema caches, keyed by the schema's absolute path. Keyed rather than
#: global because each plugin declares its own defaults; keyed rather than
#: per-class because the helper has no class to hang it on.
_SCHEMA_FONT_SIZE_CACHE: Dict[str, Dict[str, int]] = {}
def crisp_size(font_file, desired, aliases=None, grid_table=None):
"""Snap *desired* to the nearest size *font_file* renders crisply at.
A face with no known grid is returned unchanged, so a user-supplied
font is never second-guessed.
``aliases`` and ``grid_table`` default to the shared tables; a plugin
that ships an extra face can pass its own without forking this.
"""
aliases = FONT_NAME_ALIASES if aliases is None else aliases
grid_table = FONT_PIXEL_GRID if grid_table is None else grid_table
font_file = aliases.get(font_file, font_file)
grid = grid_table.get(font_file)
if not grid or not desired or desired <= 0:
return desired
return max(grid, int(round(float(desired) / grid)) * grid)
def schema_font_size(schema_path: str, element_key) -> Optional[int]:
"""The font_size this plugin's config_schema.json declares, or None.
Cached per schema path. The plugins cached this on their own class; the
path is the same distinction expressed without one, so two plugins never
share an entry.
"""
if not element_key:
return None
cache = _SCHEMA_FONT_SIZE_CACHE.get(schema_path)
if cache is None:
cache = {}
try:
import json
with open(schema_path) as fh:
schema = json.load(fh)
props = (schema.get('properties', {})
.get('customization', {})
.get('properties', {}))
for key, spec in props.items():
size = spec.get('properties', {}).get('font_size', {}).get('default')
if size is not None:
cache[key] = int(size)
except Exception:
cache = {}
_SCHEMA_FONT_SIZE_CACHE[schema_path] = cache
return cache.get(element_key)
def resolve_font_size(schema_path: str, element_config, element_key,
default_size, font_name, aliases=None, grid_table=None):
"""Size to render at: the user's choice, or a grid-snapped default.
A configured size counts as a real choice only when it differs from
the schema default. The web UI writes the whole schema default block
on every save, so "font_size == schema default" carries no intent and
would otherwise pin every install to an anti-aliased size forever.
"""
configured = (element_config or {}).get('font_size')
if configured is not None:
try:
configured = int(configured)
if configured != schema_font_size(schema_path, element_key):
return configured
except (TypeError, ValueError):
pass
return crisp_size(font_name, default_size, aliases, grid_table)
def unshare_element_fonts(logger, fonts):
"""Give each colourable element its own face object.
The colour a draw gets is resolved from the face it was handed, and
several of these loaders legitimately hand one object to more than one
element -- a size resolver that lands two elements on the same face, a
fallback that fills every key from one default, football's narrowing
step that deliberately shrinks the clock along with the score. Sharing
the object makes the element ambiguous and the colour unresolvable.
Re-instantiating from the same path and size gives a distinct object
with identical metrics, so nothing about the rendering changes; only
the ability to tell two elements apart does. Faces that cannot be
rebuilt (a BDF loaded through freetype.Face, anything without a usable
path) are left shared, and their draws stay white as before.
"""
try:
from PIL import ImageFont as _IF
except ImportError: # pragma: no cover
return fonts
seen = {}
for key in ELEMENT_FOR_FONT:
font = fonts.get(key)
if font is None:
continue
if id(font) not in seen:
seen[id(font)] = key
continue
path, size = getattr(font, "path", None), getattr(font, "size", None)
if not path or not size:
continue
try:
fonts[key] = _IF.truetype(path, size)
except (OSError, ValueError, TypeError):
logger.debug(
"Could not un-share the %s face; it keeps the default colour", key)
return fonts
-261
View File
@@ -1,261 +0,0 @@
"""The scroll/Vegas card geometry the sports scoreboards all share.
Eight scoreboards -- afl, baseball, basketball, football, hockey, lacrosse,
nrl and soccer -- each carried their own ``game_renderer.py``. After the card
helpers moved to ``src/common/sports_card.py`` the settings lookups were
shared, but the *geometry* was not: nine methods that decide how wide the
centre gap is, how much room a logo gets, and where an upcoming card's date
and time land were still eight separate copies. Five were byte-identical
across all eight plugins; the other four were identical in seven, with a
different single outlier each time.
That last detail is why this is a mixin rather than free functions. There is
no per-sport branching to write -- baseball needs its own
``_draw_upcoming_game_status`` and ``_logo_slot_width``, hockey its own
``_upcoming_date_and_time``, football its own ``_score_reserve_width``, and
every one of those is an ordinary override. The seven that agree inherit and
say nothing.
It is deliberately *only* a mixin: no ``__init__``, no state of its own. The
plugins' constructors differ in six ways and none of that difference is worth
unifying, so adoption is one line on the class statement plus deleting the
methods that now come from here.
What a host class must provide
------------------------------
Attributes: ``display_width``, ``display_height``, ``config``, ``fonts``,
``logger``, ``_team_rankings_cache``.
Methods: ``_draw_text_with_outline(draw, text, position, font, fill=None,
outline_color=(0, 0, 0))`` -- the one hook whose body genuinely varies -- plus
the ``sports_card`` delegations ``_scroll_card_option``,
``_upcoming_center_mode``, ``_vs_text``, ``_element_color``,
``_format_game_date`` and ``_format_game_time``.
"""
import math
from typing import ClassVar, Dict, Tuple
from PIL import Image, ImageDraw
class SportsGameRendererMixin:
"""Shared card geometry for the sports scoreboards. See module docstring."""
#: Centre strip as a fraction of card width, before clamping.
CENTER_GAP_RATIO: ClassVar[float] = 0.28
#: Clamp floor for the derived centre gap.
CENTER_GAP_MIN_PX: ClassVar[int] = 22
#: Clamp ceiling for the derived centre gap.
CENTER_GAP_MAX_PX: ClassVar[int] = 40
#: Breathing room kept between the score and each logo.
_SCORE_LOGO_GUTTER_PX: ClassVar[int] = 4
#: Widest score the centre strip must fit. Leagues that can reach three
#: digits a side override this with "000-000".
_SCORE_PROBE: ClassVar[str] = "00-00"
# ---- geometry ------------------------------------------------------
# Non-finite settings are rejected before any int()/round(): "inf" reaches
# these from config as a float or a string, passes an `isinstance` plus
# `>= 0` check unharmed, and then raises OverflowError out of int() --
# which the old `except (TypeError, ValueError)` did not catch, so it
# aborted the whole card render. Present in all eight plugins before this
# moved to the core; fixing it here fixes it in all eight.
def _score_reserve_width(self) -> int:
"""Centre strip the score actually needs, measured rather than assumed.
The gap was derived from the card width alone (width x
CENTER_GAP_RATIO, clamped to CENTER_GAP_MAX_PX) while the score's size
comes from config and the element-style resolver. Nothing compared the
two, so any score wider than the clamp was drawn over the logos.
Measuring it keeps the strip wide enough for whatever font is in play.
"""
try:
probe = ImageDraw.Draw(Image.new("RGB", (4, 4)))
width = probe.textlength(self._SCORE_PROBE, font=self.fonts['score'])
return int(width) + 2 * self._SCORE_LOGO_GUTTER_PX
except Exception:
self.logger.debug("Score reserve measurement failed", exc_info=True)
return 0
def _center_gap_width(self) -> int:
"""Width of the middle strip kept clear of logos.
``scroll_card.center_gap`` pins it outright; otherwise it scales with
the card width between the configurable min and max. 0 restores
edge-to-edge logos.
"""
configured = self._scroll_card_option("center_gap")
if (isinstance(configured, (int, float))
and math.isfinite(configured) and configured >= 0):
return int(configured)
ratio = self._scroll_card_option("center_gap_ratio", self.CENTER_GAP_RATIO)
low = self._scroll_card_option("center_gap_min", self.CENTER_GAP_MIN_PX)
high = self._scroll_card_option("center_gap_max", self.CENTER_GAP_MAX_PX)
try:
ratio, low, high = float(ratio), float(low), float(high)
if not (math.isfinite(ratio) and math.isfinite(low)
and math.isfinite(high)):
return self.CENTER_GAP_MIN_PX
scaled = round(self.display_width * ratio)
derived = int(max(int(low), min(int(high), scaled)))
# A strip narrower than the score is the bug, not a style choice.
# An explicit ``center_gap`` is still honoured above, including 0.
return max(derived, self._score_reserve_width())
except (TypeError, ValueError, OverflowError):
return self.CENTER_GAP_MIN_PX
def _logo_slot_width(self) -> int:
"""Per-side logo slot, leaving the center gap clear.
No longer capped at display_height: the card is sized as two
full-height logos plus the measured gap, so what is left after the gap
is exactly the logo's share. The cap was what froze the logos at 46px
on the old flat 128px card.
"""
available = (self.display_width - self._center_gap_width()) // 2
return max(8, available)
def _logo_cache_key(self, name: str) -> str:
"""Cache key scoped to the logo slot.
One cache dict is shared by renderers built for different card widths,
so a logo sized for a wide slot must not be handed to a narrow one.
"""
return f"{name}@{self._logo_slot_width()}x{self.display_height}"
def _layout_offset(self, element: str, axis: str, default: int = 0) -> int:
"""X/Y nudge for one element, from customization.layout.
Same block the full-screen scorebug reads (sports.py
_get_layout_offset), so a nudge configured in the web UI now moves
the element on the scroll/Vegas card too -- previously the schema
advertised these offsets but this renderer ignored them.
"""
try:
layout = (self.config or {}).get("customization", {}).get("layout", {})
value = (layout.get(element) or {}).get(axis, default)
if isinstance(value, bool):
return default
if isinstance(value, (int, float)):
return int(value) if math.isfinite(value) else default
if isinstance(value, str):
parsed = float(value)
return int(parsed) if math.isfinite(parsed) else default
except (TypeError, ValueError, OverflowError):
pass
return default
# ---- upcoming cards ------------------------------------------------
def _upcoming_date_and_time(self, game: Dict) -> Tuple[str, str]:
"""(date, time) for an upcoming card, from the extractor's flat keys."""
return (
str(game.get("game_date", "") or ""),
str(game.get("game_time", "") or ""),
)
def _draw_upcoming_center(self, draw: "ImageDraw.ImageDraw", game: Dict) -> None:
"""Draw the middle of an upcoming card.
Never a score: an upcoming game has not started, so the extractor's
0-0 is noise. Either the VS text (default), the date and time stacked,
or nothing at all.
"""
mode = self._upcoming_center_mode()
if mode == "none":
return
if mode == "vs":
vs_text = self._vs_text()
if not vs_text:
return
vs_width = draw.textlength(vs_text, font=self.fonts['score'])
vs_x = (self.display_width - vs_width) // 2 + self._layout_offset('score', 'x_offset')
vs_y = (self.display_height // 2) - 3 + self._layout_offset('score', 'y_offset')
self._draw_text_with_outline(
draw, vs_text, (vs_x, vs_y), self.fonts['score'],
fill=self._element_color('score_text')
)
return
date_text, time_text = self._upcoming_date_and_time(game)
lines = []
if self._scroll_card_option("show_date", True):
lines.append(self._format_game_date(date_text, game))
if self._scroll_card_option("show_time", True):
lines.append(self._format_game_time(time_text))
lines = [t for t in lines if t]
if not lines:
return
font = self.fonts.get('detail') or self.fonts['time']
line_h = 7
top = (self.display_height // 2) - (len(lines) * line_h) // 2
top += self._layout_offset('score', 'y_offset')
for i, line in enumerate(lines):
width = draw.textlength(line, font=font)
x = (self.display_width - width) // 2 + self._layout_offset('score', 'x_offset')
self._draw_text_with_outline(
draw, line, (x, top + i * line_h), font,
fill=self._element_color('detail_text')
)
def _draw_upcoming_game_status(self, draw: "ImageDraw.ImageDraw", game: Dict) -> None:
"""Draw the date and time around an upcoming card.
Time top and date bottom by default; scroll_card.swap_date_time puts
the date on top instead. Skipped when the pair is stacked in the
middle, which would otherwise print them twice.
"""
if self._upcoming_center_mode() == "date_time":
return
date_raw, time_raw = self._upcoming_date_and_time(game)
date_text = (self._format_game_date(date_raw, game)
if self._scroll_card_option("show_date", True) else "")
time_text = (self._format_game_time(time_raw)
if self._scroll_card_option("show_time", True) else "")
if self._scroll_card_option("swap_date_time", False):
top_text, top_el, bottom_text, bottom_el = (
date_text, 'date', time_text, 'time')
top_font = self.fonts.get('detail') or self.fonts['time']
bottom_font = self.fonts['time']
top_color, bottom_color = 'detail_text', 'period_text'
else:
top_text, top_el, bottom_text, bottom_el = (
time_text, 'time', date_text, 'date')
top_font = self.fonts['time']
bottom_font = self.fonts.get('detail') or self.fonts['time']
top_color, bottom_color = 'period_text', 'detail_text'
if top_text:
top_width = draw.textlength(top_text, font=top_font)
top_x = (self.display_width - top_width) // 2 + self._layout_offset(top_el, 'x_offset')
top_y = 1 + self._layout_offset(top_el, 'y_offset')
self._draw_text_with_outline(
draw, top_text, (top_x, top_y), top_font,
fill=self._element_color(top_color)
)
if bottom_text:
bottom_width = draw.textlength(bottom_text, font=bottom_font)
bottom_x = ((self.display_width - bottom_width) // 2
+ self._layout_offset(bottom_el, 'x_offset'))
# Measured, not a fixed -7: the detail font is 6px in most plugins
# but 10px in soccer and nrl, where "Sep 19" ran past the card.
ink_bottom = draw.textbbox((0, 0), bottom_text, font=bottom_font)[3]
bottom_y = (max(0, self.display_height - ink_bottom - 1)
+ self._layout_offset(bottom_el, 'y_offset'))
self._draw_text_with_outline(
draw, bottom_text, (bottom_x, bottom_y), bottom_font,
fill=self._element_color(bottom_color)
)
# ---- rankings ------------------------------------------------------
def set_rankings_cache(self, rankings: Dict[str, int]) -> None:
"""Set the team rankings cache for display."""
self._team_rankings_cache = rankings
File diff suppressed because it is too large Load Diff
+8 -84
View File
@@ -181,16 +181,6 @@ class DisplayController:
self.plugin_modes = {} # mode -> plugin_instance mapping for plugin-first dispatch
self.mode_to_plugin_id: Dict[str, str] = {}
self.plugin_display_modes: Dict[str, List[str]] = {}
# plugin_display_modes is mutated only by _register_loaded_plugin /
# _unregister_plugin on the render thread, but the config-watcher
# thread reads it in _enabled_plugin_not_running. Both mutation sites
# run during reconcile (rare), so this lock never touches the per-frame
# path -- the hot-path reads are same-thread as the writes.
self._plugin_modes_lock = threading.Lock()
# Guards the consume-and-clear of _pending_plugin_reconcile. Only taken
# when a reconcile is actually pending or a config change arrives, both
# rare -- the per-frame path just reads the bool.
self._reconcile_flag_lock = threading.Lock()
# Per-plugin config-change callbacks, kept so we can unsubscribe a
# plugin when it is disabled live.
self._plugin_config_callbacks: Dict[str, Callable] = {}
@@ -473,10 +463,8 @@ class DisplayController:
self._refresh_config_cache(new_config)
# If a plugin was enabled/disabled, flag a reconcile for the main
# loop to apply (loading/unloading off the watcher thread is unsafe).
if (self._enabled_set_changed(old_config, new_config)
or self._enabled_plugin_not_running(new_config)):
with self._reconcile_flag_lock:
self._pending_plugin_reconcile = True
if self._enabled_set_changed(old_config, new_config):
self._pending_plugin_reconcile = True
self.config_service.subscribe(_controller_config_change)
@@ -1761,12 +1749,11 @@ class DisplayController:
# rebuilding available_modes happens here on the render thread so
# it can't race with rendering. Deferred while on-demand is active
# (the flag stays set) so we don't fight its temporary-enable.
# The lock-free read is a fast path only; it can be a false
# negative (the watcher setting the flag just after it is read
# is seen next iteration), never a false positive that loses a
# request.
if self._pending_plugin_reconcile and not self.on_demand_active:
self._service_pending_reconcile()
# Only clear the flag on success -- a retryable failure
# (e.g. discovery) leaves it set so the request isn't lost.
if self._reconcile_enabled_plugins():
self._pending_plugin_reconcile = False
if not self.available_modes:
# Nothing to render yet. Re-check _pending_plugin_reconcile
@@ -2826,8 +2813,7 @@ class DisplayController:
logger.debug("Using manifest display_modes for %s: %s", plugin_id, display_modes)
if not (isinstance(display_modes, list) and display_modes):
display_modes = [plugin_id]
with self._plugin_modes_lock:
self.plugin_display_modes[plugin_id] = list(display_modes)
self.plugin_display_modes[plugin_id] = list(display_modes)
# Subscribe to config changes for per-plugin hot-reload. Bind plugin_id
# and instance as defaults so each plugin's callback targets its own
@@ -2861,8 +2847,7 @@ class DisplayController:
def _unregister_plugin(self, plugin_id: str) -> None:
"""Remove a plugin's modes, config subscription and instance, then
unload it. Used by live disable hot-reload."""
with self._plugin_modes_lock:
modes = self.plugin_display_modes.pop(plugin_id, [])
modes = self.plugin_display_modes.pop(plugin_id, [])
for mode in modes:
if mode in self.available_modes:
self.available_modes.remove(mode)
@@ -2907,67 +2892,6 @@ class DisplayController:
}
return enabled_map(old_config) != enabled_map(new_config)
def _service_pending_reconcile(self) -> None:
"""Consume a pending reconcile request and run it.
The request is consumed BEFORE reconciling, not cleared after. Clearing
after would drop any config change that lands while reconcile is
running: reconcile has already read its config by then, so the clear
erases a request it never served and the newest config never
reconciles -- the same "your save did nothing" failure this whole path
exists to prevent. Consuming first means such a request stays set and
is picked up on the next pass.
A retryable failure (e.g. discovery) re-arms the flag.
"""
with self._reconcile_flag_lock:
pending = self._pending_plugin_reconcile
self._pending_plugin_reconcile = False
if pending and not self._reconcile_enabled_plugins():
with self._reconcile_flag_lock:
self._pending_plugin_reconcile = True
def _enabled_plugin_not_running(self, new_config: Dict[str, Any]) -> bool:
"""True when a discovered plugin is enabled in config but not running.
``_enabled_set_changed`` compares only top-level ``enabled`` flags, which
misses the case that strands a plugin: one whose ``validate_config()``
returned False is absent from the running set, and the edit that fixes it
(enabling a league, filling in an API key) lives *nested* inside that
plugin's own section. No top-level flag changes, so no reconcile is
queued, and the save that should have fixed it appears to do nothing --
only toggling some unrelated plugin recovers it. hockey-scoreboard sat
enabled-but-absent on a live rig for four days this way.
Deliberately narrow: it fires only for ids the plugin manager has
actually discovered, so non-plugin sections that carry their own
``enabled`` flag (``schedule``, ``display``, ...) don't queue a reconcile
on every save. In the steady state -- everything enabled is loaded --
this is False and costs nothing. That matters because reconcile runs
``discover_plugins()`` on the render thread, where a needless
filesystem scan per config save would show up as a frame hitch.
Runs on the config-watcher thread, so both mappings it reads are
snapshotted under the lock that guards their writes.
"""
if self.plugin_manager is None:
return False
# Two snapshots, each taken under its own lock and never nested, so a
# half-written mapping is never observed and this can't deadlock
# against discovery (which holds the discovery lock while rebuilding).
try:
known = self.plugin_manager.discovered_plugin_ids()
except AttributeError:
# Older manager without the accessor: fall back to a plain read.
known = set(getattr(self.plugin_manager, 'plugin_manifests', ()) or ())
with self._plugin_modes_lock:
running = set(self.plugin_display_modes)
for key, value in new_config.items():
if (key in known and isinstance(value, dict)
and value.get('enabled', False) and key not in running):
return True
return False
def _reconcile_enabled_plugins(self) -> bool:
"""Load/unload plugins so the running set matches the enabled set in
config. Runs on the main display thread (never the config-watcher
+8 -116
View File
@@ -14,7 +14,6 @@ import json
from typing import Dict, List, Optional, Tuple
from pathlib import Path
from PIL import Image, ImageDraw, ImageFont
from PIL.PngImagePlugin import PngInfo
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from src.common.permission_utils import (
@@ -26,96 +25,6 @@ from src.common.permission_utils import (
logger = logging.getLogger(__name__)
#: PNG text key stamped into a generated placeholder so a later run can tell it
#: apart from a real logo that happens to be small.
PLACEHOLDER_MARKER = "ledmatrix_placeholder"
#: Geometry of a generated placeholder, used to recognise ones written before
#: the marker existed. Those are already on users' disks and would otherwise
#: never be retried.
PLACEHOLDER_SIZE = (64, 64)
PLACEHOLDER_BG = (100, 100, 100, 255)
#: How long a placeholder is trusted before the real logo is attempted again.
#: A placeholder means the download failed, and download failures are usually
#: transient (no network at boot, ESPN blipping). Retrying every frame would
#: hammer the API from a Pi that is also driving a panel; never retrying leaves
#: the team a grey box forever, which is the bug this exists to avoid.
PLACEHOLDER_RETRY_SECONDS = 6 * 60 * 60
def is_placeholder_logo(filepath: Path) -> bool:
"""True if the file at ``filepath`` is a generated placeholder, not a logo.
Checks the marker first, then falls back to matching the placeholder's
exact geometry and background colour so files written before the marker was
introduced are still recognised.
"""
try:
with Image.open(filepath) as img:
if img.info.get(PLACEHOLDER_MARKER):
return True
if img.size != PLACEHOLDER_SIZE:
return False
return img.convert("RGBA").getpixel((0, 0)) == PLACEHOLDER_BG
except Exception:
# Unreadable file: not provably a placeholder, and the caller's own
# error handling is better placed to deal with it.
return False
def should_attempt_download(filepath: Path, force_download: bool = False) -> bool:
"""Whether a real logo is worth (re)fetching for ``filepath``.
True when nothing is there, when the caller forced it, or when what is
there is a placeholder old enough to retry. A *fresh* placeholder says a
download just failed, so retrying it immediately would hammer the API for
a result that is very unlikely to have changed.
"""
if force_download or not filepath.exists():
return True
if not is_placeholder_logo(filepath):
return False
age = placeholder_age_seconds(filepath)
return age is None or age >= PLACEHOLDER_RETRY_SECONDS
def refresh_placeholder_timestamp(filepath: Path) -> bool:
"""Restamp a placeholder so a failed retry restarts the back-off clock.
Without this a stale placeholder stays stale: every later call sees an
expired timestamp, retries, fails, and leaves the timestamp untouched --
which is a download attempt per call, the opposite of what the back-off is
for.
"""
try:
if not is_placeholder_logo(filepath):
return False
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time()))
with Image.open(filepath) as img:
img.copy().save(filepath, "PNG", pnginfo=metadata)
return True
except Exception:
logger.debug("Could not refresh placeholder timestamp for %s", filepath,
exc_info=True)
return False
def placeholder_age_seconds(filepath: Path) -> Optional[float]:
"""Seconds since a placeholder was written, or None if unknown."""
try:
with Image.open(filepath) as img:
stamped = img.info.get(PLACEHOLDER_MARKER)
if stamped and stamped != "1":
return max(0.0, time.time() - float(stamped))
except Exception:
pass
try:
return max(0.0, time.time() - filepath.stat().st_mtime)
except OSError:
return None
class LogoDownloader:
"""Centralized logo downloader for team logos from ESPN API."""
@@ -590,10 +499,8 @@ class LogoDownloader:
filename = f"{self.normalize_abbreviation(abbreviation)}.png"
filepath = Path(logo_dir) / filename
# A placeholder does not count as existing -- it is a previous
# failure, and a bulk pass is exactly where it should get another
# chance, subject to the same back-off as everywhere else.
if not should_attempt_download(filepath, force_download):
# Skip if already exists and not forcing download
if filepath.exists() and not force_download:
logger.debug(f"Skipping {display_name}: {filename} already exists")
continue
@@ -652,9 +559,8 @@ class LogoDownloader:
filename = f"{self.normalize_abbreviation(abbreviation)}.png"
filepath = Path(logo_dir) / filename
# Same eligibility rule as every other download site: a stale
# placeholder is a failed download, not a logo.
if not should_attempt_download(filepath, force_download):
# Skip if already exists and not forcing download
if filepath.exists() and not force_download:
logger.debug(f"Skipping {display_name} ({category}, {conference}): {filename} already exists")
continue
@@ -768,16 +674,11 @@ class LogoDownloader:
# Fallback without font
draw.text((16, 24), text, fill=(255, 255, 255, 255))
# Stamp it so a later run can tell this apart from a real logo and
# retry the download, instead of treating the file's existence as
# proof the logo was fetched.
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time()))
logo.save(filepath, "PNG", pnginfo=metadata)
logo.save(filepath)
# Set proper file permissions after saving
ensure_file_permissions(filepath, get_assets_file_mode())
logger.info(f"Created placeholder logo for {team_abbreviation} at {filepath}")
return True
@@ -869,18 +770,9 @@ def download_missing_logo(league: str, team_id: str, team_abbreviation: str, log
# Use the exact filepath that was passed in (respects config settings)
filepath = logo_path
if filepath.exists() and not should_attempt_download(filepath):
# Either a real logo, or a placeholder too fresh to be worth retrying.
if filepath.exists():
logger.debug(f"Logo already exists for {team_abbreviation} ({league})")
return True
if filepath.exists():
# A placeholder is a *failed* download wearing the real logo's
# filename. Treating it as "already exists" is what pinned a team to a
# grey box permanently after one transient failure.
logger.info(
"Logo for %s (%s) is a placeholder from a failed download; "
"retrying the real logo", team_abbreviation, league,
)
# Try to download the real logo first
logger.info(f"Attempting to download logo for {team_abbreviation} from {league}")
-11
View File
@@ -631,17 +631,6 @@ class PluginManager:
return self.load_plugin(plugin_id)
def discovered_plugin_ids(self) -> set:
"""Snapshot of the discovered plugin ids, taken under the discovery lock.
Callers on other threads (the config watcher) must not iterate
``plugin_manifests`` directly: discovery rebuilds it entry by entry, so
an unsynchronised reader can see a half-populated mapping or raise
"dictionary changed size during iteration".
"""
with self._discovery_lock:
return set(self.plugin_manifests)
def get_plugin(self, plugin_id: str) -> Optional[Any]:
"""
Get a loaded plugin instance by ID.
+33 -112
View File
@@ -6,40 +6,14 @@ with state transitions and queries.
"""
import threading
import time
from collections import deque
from enum import Enum
from typing import Optional, Dict, Any, Deque, List, Tuple
from typing import Optional, Dict, Any
from datetime import datetime
import logging
from src.logging_config import get_logger
# The history is diagnostic only -- nothing reads the entries themselves, just
# their count -- but it is appended to on the hot scheduling path: every update
# cycle records RUNNING on reserve and ENABLED on finish. Unbounded, that is
# 2,880 entries per plugin per day at the default 60s interval, which on a 1 GB
# Pi exhausts memory in weeks.
#
# Two limits, because a single entry count answers the wrong question. What a
# reader wants is "the last couple of hours", and how many transitions that is
# depends entirely on the plugin's update interval -- which on a real board
# spans 2s to 3600s. A flat 200 entries is 4.2 days for the slowest plugin and
# 3.3 minutes for the fastest, so the plugin churning hardest, the one worth
# looking at, keeps the least history.
#
# So: trim by AGE first, which makes the retained window comparable across
# plugins whatever their cadence...
STATE_HISTORY_MAX_AGE_SECONDS = 2 * 60 * 60
# ...and cap by COUNT second, purely as a memory ceiling for the fast pollers
# whose age window would otherwise run to thousands of entries. At ~230 bytes
# an entry this is ~0.5 MB per plugin worst case, and only plugins updating
# faster than roughly every 4s can reach it.
MAX_STATE_HISTORY_PER_PLUGIN = 2000
class PluginState(Enum):
"""Plugin state enumeration."""
UNLOADED = "unloaded" # Plugin not loaded
@@ -63,43 +37,11 @@ class PluginStateManager:
self.logger = logger or get_logger(__name__)
self._lock = threading.RLock()
self._states: Dict[str, PluginState] = {}
# (monotonic timestamp, transition). The clock is monotonic so a DST
# shift or an NTP step cannot make entries look old and flush the
# history; the human-readable timestamp lives inside the transition.
self._state_history: Dict[str, Deque[Tuple[float, Dict[str, Any]]]] = {}
# Lifetime transition totals, kept separately so the count reported by
# get_state_info() stays truthful once the history above starts rolling.
self._state_transition_counts: Dict[str, int] = {}
self._state_history: Dict[str, list] = {}
self._error_info: Dict[str, Dict[str, Any]] = {}
self._last_update: Dict[str, datetime] = {}
self._last_display: Dict[str, datetime] = {}
def _record_transition(
self,
plugin_id: str,
transition: Dict[str, Any]
) -> None:
"""Append a transition to the plugin's bounded history.
Callers must already hold ``_lock``. The deque discards its oldest
entry once it is full, so the history cannot grow without bound; the
lifetime total is tracked separately for get_state_info().
"""
history = self._state_history.get(plugin_id)
if history is None:
history = deque(maxlen=MAX_STATE_HISTORY_PER_PLUGIN)
self._state_history[plugin_id] = history
now = time.monotonic()
history.append((now, transition))
# Age out first; the deque's maxlen is the backstop for plugins that
# produce more than the ceiling within the window.
cutoff = now - STATE_HISTORY_MAX_AGE_SECONDS
while history and history[0][0] < cutoff:
history.popleft()
self._state_transition_counts[plugin_id] = (
self._state_transition_counts.get(plugin_id, 0) + 1
)
def set_state(
self,
plugin_id: str,
@@ -118,13 +60,16 @@ class PluginStateManager:
old_state = self._states.get(plugin_id, PluginState.UNLOADED)
self._states[plugin_id] = state
if plugin_id not in self._state_history:
self._state_history[plugin_id] = []
transition = {
'timestamp': datetime.now(),
'from': old_state.value,
'to': state.value,
'error': str(error) if error else None
}
self._record_transition(plugin_id, transition)
self._state_history[plugin_id].append(transition)
# Store error info if transitioning to ERROR state
if state == PluginState.ERROR and error:
@@ -181,29 +126,17 @@ class PluginStateManager:
state = self.get_state(plugin_id)
return state == PluginState.ENABLED
def get_state_history(self, plugin_id: str) -> List[Dict[str, Any]]:
def get_state_history(self, plugin_id: str) -> list:
"""
Get state transition history for a plugin.
Retention is by age first -- transitions older than
STATE_HISTORY_MAX_AGE_SECONDS are dropped -- and by count second, at
MAX_STATE_HISTORY_PER_PLUGIN, which only binds for plugins updating
fast enough to exceed it inside that window.
Args:
plugin_id: Plugin identifier
Returns:
List of recent state transitions, oldest first. Both the list and
the transition dicts are copies, so callers cannot mutate the
manager's own history. The values inside a transition are all
immutable, so a shallow copy per entry is enough.
List of state transitions
"""
with self._lock:
return [
dict(transition)
for _stamp, transition in self._state_history.get(plugin_id, ())
]
return self._state_history.get(plugin_id, [])
def set_error_info(self, plugin_id: str, error_info: Dict[str, Any]) -> None:
"""
@@ -246,7 +179,9 @@ class PluginStateManager:
old_state = self._states.get(plugin_id, PluginState.UNLOADED)
self._states[plugin_id] = state
self._record_transition(plugin_id, {
if plugin_id not in self._state_history:
self._state_history[plugin_id] = []
self._state_history[plugin_id].append({
'timestamp': datetime.now(),
'from': old_state.value,
'to': state.value,
@@ -306,40 +241,26 @@ class PluginStateManager:
Returns:
Dictionary with state information
"""
# One snapshot, one critical section. Each field was read under its own
# lock, so an unload running concurrently could be observed half-done:
# 'state' read before clear_state() removed it and
# 'state_history_count' read after, giving a caller a plugin that is
# ENABLED with zero transitions. _lock is an RLock, so the helpers
# below can still take it.
with self._lock:
state = self.get_state(plugin_id)
info = {
'state': state.value,
'is_loaded': self.is_loaded(plugin_id),
'is_enabled': self.is_enabled(plugin_id),
'is_running': self.is_running(plugin_id),
'is_error': self.is_error(plugin_id),
'can_execute': self.can_execute(plugin_id),
'last_update': self.get_last_update(plugin_id),
'last_display': self.get_last_display(plugin_id),
'error_info': self.get_error_info(plugin_id),
'state_history_count': self._state_transition_counts.get(plugin_id, 0)
}
state = self.get_state(plugin_id)
info = {
'state': state.value,
'is_loaded': self.is_loaded(plugin_id),
'is_enabled': self.is_enabled(plugin_id),
'is_running': self.is_running(plugin_id),
'is_error': self.is_error(plugin_id),
'can_execute': self.can_execute(plugin_id),
'last_update': self.get_last_update(plugin_id),
'last_display': self.get_last_display(plugin_id),
'error_info': self.get_error_info(plugin_id),
'state_history_count': len(self.get_state_history(plugin_id))
}
return info
def clear_state(self, plugin_id: str) -> None:
"""Clear all state information for a plugin.
Held under ``_lock`` so the five dicts are dropped as one unit: every
other mutator takes the lock, and without it a concurrent set_state()
could interleave and leave a plugin with history but no state.
"""
with self._lock:
self._states.pop(plugin_id, None)
self._state_history.pop(plugin_id, None)
self._state_transition_counts.pop(plugin_id, None)
self._error_info.pop(plugin_id, None)
self._last_update.pop(plugin_id, None)
self._last_display.pop(plugin_id, None)
"""Clear all state information for a plugin."""
self._states.pop(plugin_id, None)
self._state_history.pop(plugin_id, None)
self._error_info.pop(plugin_id, None)
self._last_update.pop(plugin_id, None)
self._last_display.pop(plugin_id, None)
-134
View File
@@ -1602,26 +1602,6 @@ class PluginStoreManager:
'error': f'Manifest missing required fields: {", ".join(missing_fields)}'
}
# Refuse a plugin that needs a newer core than this one, exactly as
# _install_plugin_impl does after its download. Sideloading is an
# explicit act rather than an automatic store update, but the floor
# is not advice about intent -- it is a statement that the plugin
# cannot run here, and letting it through produces the same silent
# PluginState.ERROR at load. This was the last of the three routes
# in that skipped the check.
#
# Before the move, so the `finally` below removes the temp tree and
# nothing half-installed is left behind.
from src import __version__ as core_version
from src.plugin_system import compatibility
compatible, reason = compatibility.check(manifest, core_version)
if not compatible:
self.logger.error(
"Refusing to install %s from %s: %s",
plugin_id, repo_url, reason)
return {'success': False, 'error': reason}
# Validate version fields consistency (warnings only, not required)
validation_errors = self._validate_manifest_version_fields(manifest)
if validation_errors:
@@ -2603,85 +2583,6 @@ class PluginStoreManager:
self.logger.error(f"Error uninstalling plugin {plugin_id}: {e}")
return False
def _gate_pulled_commit(self, plugin_id: str, plugin_path: Path,
previous_sha: Optional[str]) -> bool:
"""Apply the compatibility gate to a commit that arrived via git pull.
Every other route into an installed plugin goes through
``install_plugin``, which gates in ``_install_plugin_impl``. This one
did not: a ``git pull`` could deliver a manifest flooring above this
core and nothing would notice until the plugin failed to load, which
surfaces as one line in the journal and a scoreboard that silently
stopped appearing.
Checked after the pull rather than before it, for the same reason
``_install_plugin_impl`` checks after the download: the registry
carries no compatibility field, so the incoming floor is only knowable
once the new commit is on disk.
Undone with ``git reset --hard`` rather than by removing the directory.
This is a live checkout, the previous commit is still in the object
store, and the reset leaves the user on the exact version they were
already running -- the same promise ``_reinstall_with_rollback`` makes,
reached by the means this path actually has. It is also the gentler
option: no window in which the plugin directory does not exist, and no
``.standalone-backup-`` debris if the process dies mid-way.
A manifest that cannot be read is not evidence of incompatibility, so
it allows. ``compatibility.check`` refuses only on evidence for the
same reason: a wrong refusal breaks a working install, while a wrong
allowance degrades to exactly the behaviour this path had before the
gate existed.
"""
manifest_path = plugin_path / "manifest.json"
try:
with open(manifest_path, 'r', encoding='utf-8') as mf:
manifest = json.load(mf)
except (OSError, ValueError) as e:
self.logger.warning(
"Could not read %s after updating %s (%s); allowing the "
"update, as an unreadable manifest declares no floor",
manifest_path, plugin_id, e)
return True
from src import __version__ as core_version
from src.plugin_system import compatibility
compatible, reason = compatibility.check(manifest, core_version)
if compatible:
return True
self.logger.error("Refusing the update to %s: %s", plugin_id, reason)
if not previous_sha:
self.logger.error(
"Cannot roll %s back: the commit it was on before the pull is "
"unknown. It is now on a version this core cannot run — "
"reinstall it from the plugin store.", plugin_id)
return False
# Safe by construction: update_plugin returns before pulling unless the
# tree was clean or successfully stashed, so there are no uncommitted
# tracked edits for --hard to discard. The stash is not popped on the
# success path either, so the reset leaves the working tree exactly
# where a successful pull would have. Say "commit", not "changes".
reset = subprocess.run(
['git', '-C', str(plugin_path), 'reset', '--hard', previous_sha],
capture_output=True, text=True, timeout=60, check=False)
if reset.returncode != 0:
self.logger.error(
"CRITICAL: could not roll %s back to commit %s: %s. It is left "
"on a version this core cannot run; "
"`git -C %s reset --hard %s` restores it.",
plugin_id, previous_sha[:7],
(reset.stderr or reset.stdout or '').strip(),
plugin_path, previous_sha)
else:
self.logger.info(
"Rolled %s back to commit %s; it stays on the version it was "
"already running.", plugin_id, previous_sha[:7])
return False
def _reinstall_with_rollback(self, plugin_id: str, plugin_path: Path) -> bool:
"""Replace an installed plugin with a fresh install, atomically.
@@ -2959,8 +2860,6 @@ class PluginStoreManager:
status_result = type('obj', (object,), {'stdout': '', 'stderr': 'Status check timed out'})()
stash_info = ""
# Whether the pull can be undone without destroying work.
tree_is_recoverable = not has_changes
if has_changes:
self.logger.info(f"Stashing local changes in {plugin_id} before update")
try:
@@ -2974,37 +2873,12 @@ class PluginStoreManager:
)
if stash_result.returncode == 0:
stash_info = " (local changes were stashed)"
tree_is_recoverable = True
self.logger.info(f"Stashed local changes (including untracked files) for {plugin_id}")
else:
self.logger.warning(f"Failed to stash local changes for {plugin_id}: {stash_result.stderr}")
except subprocess.TimeoutExpired:
self.logger.warning(f"Stash operation timed out for {plugin_id}, proceeding with pull")
# Do not pull what cannot be un-pulled.
#
# The compatibility gate below can refuse the commit this
# pull brings down, and its only way back is `git reset
# --hard`, which discards uncommitted tracked edits. Those
# edits are exactly what the stash above exists to protect,
# so a stash that failed or timed out leaves the rollback
# unable to run without destroying them.
#
# A pull does not necessarily refuse on a dirty tree -- git
# merges happily as long as the incoming commit touches
# different files -- so without this the update would
# succeed, the gate would refuse, and the reset would take
# the user's work with it. Refusing here costs an update in
# a case that already went wrong; the alternative costs
# data.
if not tree_is_recoverable:
self.logger.error(
"Refusing to update %s: it has local changes that could "
"not be stashed, and an incompatible update could then "
"only be rolled back by discarding them. Commit or stash "
"them by hand, then update.", plugin_id)
return False
# Pull from the determined remote branch
self.logger.info(f"Pulling from origin/{remote_pull_branch} for {plugin_id}...")
pull_result = subprocess.run(
@@ -3027,14 +2901,6 @@ class PluginStoreManager:
elif updated_sha:
self.logger.info(f"Plugin {plugin_id} updated to commit {updated_sha[:7]}{stash_info}")
# The install gate, at the only point on this path where
# it can be answered. Every other route in goes through
# install_plugin, which gates in _install_plugin_impl; this
# one did not, so a pull could deliver a manifest flooring
# above this core and nothing would notice.
if not self._gate_pulled_commit(plugin_id, plugin_path, local_sha):
return False
self._install_dependencies(plugin_path)
return True
+11 -75
View File
@@ -31,18 +31,6 @@ if TYPE_CHECKING:
logger = logging.getLogger(__name__)
#: Degradation threshold, as a fraction of target_fps. A marquee jitters a
#: little all the time, so "anything under target" would report constantly and
#: mean nothing; 90% of target is the point where a shortfall is real. At a
#: 60fps target that is 54fps -- 55fps is a normal wobble and stays at DEBUG,
#: which is deliberate, not an off-by-one.
_FPS_HEALTHY_FRACTION = 0.9
#: A healthy marquee still reports this often, so silence means stopped
#: rather than fine.
_FPS_HEARTBEAT_INTERVAL = 300.0
def _percentile(ordered: List[float], fraction: float) -> float:
"""Nearest-rank percentile of an already-sorted list.
@@ -108,11 +96,6 @@ class VegasModeCoordinator:
self._is_active = False
self._is_paused = False
self._should_stop = False
# Frame-rate health, tracked across run_iteration() calls so the
# heartbeat is one-per-interval rather than one-per-cycle, and so a
# recovery spanning two cycles is still reported. Reset on start().
self._fps_last_health_log = 0.0
self._fps_was_degraded = False
self._state_lock = threading.Lock()
# Live priority tracking
@@ -265,11 +248,6 @@ class VegasModeCoordinator:
self._is_active = True
self._should_stop = False
self._start_time = time.time()
# A fresh run starts with a clean health slate: no stale
# "was degraded" from the previous run, and a heartbeat that is
# due immediately so the first sample confirms the marquee is up.
self._fps_last_health_log = 0.0
self._fps_was_degraded = False
# Line up the next group immediately, so the first extension is already
# warm rather than stalling the scroll to fetch it.
@@ -417,18 +395,8 @@ class VegasModeCoordinator:
duration = self.render_pipeline.get_dynamic_duration()
start_time = time.time()
frame_count = 0
fps_log_interval = 5.0 # Sample FPS every 5 seconds
# Health state lives on the coordinator, not here: run_iteration() is
# called once per cycle, so locals reset every few seconds. That made
# `last_fps_health_log = 0.0` fire the "heartbeat" on the first sample
# of every iteration rather than once per interval, and a recovery
# that crossed an iteration boundary was never reported at all --
# was_degraded had already gone back to False.
# Monotonic, and deliberately not start_time: start_time is wall
# clock and is used below to report the iteration's duration. Mixing
# the two here would make every delta hugely negative and silence the
# frame-rate reporting altogether.
last_fps_log_time = time.monotonic()
fps_log_interval = 5.0 # Log FPS every 5 seconds
last_fps_log_time = start_time
fps_frame_count = 0
# A mean hides stutter completely. At 120fps a five-second window is
# ~600 frames, so a 200ms freeze -- plainly visible on a marquee --
@@ -440,13 +408,7 @@ class VegasModeCoordinator:
logger.info("Starting Vegas iteration for %.1fs", duration)
while True:
# Monotonic, like the FPS window below. These devices have no RTC,
# so the wall clock jumps by however wrong boot time was the moment
# NTP first syncs. A backward jump makes frame_elapsed negative,
# and `frame_interval - frame_elapsed` then sleeps for longer than
# the whole budget -- the render loop stalls for the size of the
# correction. A forward jump inflates p99 and worst-frame instead.
frame_started = time.monotonic()
frame_started = time.time()
# Check for STATIC mode plugin that should pause scroll
static_plugin = self._check_static_plugin_trigger()
@@ -474,7 +436,7 @@ class VegasModeCoordinator:
# quarter of the budget spent not rendering. Subtracting the work
# already done keeps the pacing target while reclaiming that time,
# and yields the GIL either way so other threads still run.
frame_elapsed = time.monotonic() - frame_started
frame_elapsed = time.time() - frame_started
time.sleep(max(0.0, frame_interval - frame_elapsed))
# Measured before the sleep: time spent working, not pacing.
@@ -486,42 +448,16 @@ class VegasModeCoordinator:
frame_count += 1
fps_frame_count += 1
# Periodic FPS logging. Reported at INFO only when the frame rate
# is actually worth an operator's attention -- a shortfall against
# target, or the recovery from one -- with a slow heartbeat so a
# healthy marquee still shows a pulse.
#
# Measured over two hours on a running rig: 1410 samples, 98.5%
# of them within 10% of target. The 1.5% that were not included a
# reading of 8.6fps against a target of 60 -- a real stall, and
# completely invisible inside 1389 lines reading "59.6".
# Monotonic: every use of this value in the block below is a
# duration, and these devices have no RTC, so the wall clock jumps
# by however wrong boot time was the moment NTP first syncs. That
# would not only mis-fire the heartbeat, it would corrupt the
# frame rate itself, since fps is frames divided by this delta.
current_time = time.monotonic()
# Periodic FPS logging
current_time = time.time()
if current_time - last_fps_log_time >= fps_log_interval:
fps = fps_frame_count / (current_time - last_fps_log_time)
p99 = _percentile(sorted(frame_times), 0.99)
target = self.vegas_config.target_fps
degraded = target > 0 and fps < target * _FPS_HEALTHY_FRACTION
due = (current_time - self._fps_last_health_log
>= _FPS_HEARTBEAT_INTERVAL)
if degraded or self._fps_was_degraded or due:
logger.info(
"Vegas FPS: %.1f (target: %d, frames: %d) p99 %.1fms worst %.1fms",
fps, target, fps_frame_count,
p99 * 1000.0, frame_worst * 1000.0
)
self._fps_last_health_log = current_time
else:
logger.debug(
"Vegas FPS: %.1f (target: %d, frames: %d) p99 %.1fms worst %.1fms",
fps, target, fps_frame_count,
p99 * 1000.0, frame_worst * 1000.0
)
self._fps_was_degraded = degraded
logger.info(
"Vegas FPS: %.1f (target: %d, frames: %d) p99 %.1fms worst %.1fms",
fps, self.vegas_config.target_fps, fps_frame_count,
p99 * 1000.0, frame_worst * 1000.0
)
last_fps_log_time = current_time
fps_frame_count = 0
frame_worst = 0.0
+1 -79
View File
@@ -5,7 +5,7 @@ Provides functions for identifying, masking, separating, and filtering
secret fields in plugin configurations based on JSON Schema x-secret markers.
"""
from typing import Any, Dict, Optional, Set, Tuple
from typing import Any, Dict, Set, Tuple
def find_secret_fields(properties: Dict[str, Any], prefix: str = '') -> Set[str]:
@@ -202,89 +202,11 @@ def remove_empty_secrets(secrets: Dict[str, Any]) -> Dict[str, Any]:
nested = remove_empty_secrets(v)
if nested:
result[k] = nested
elif isinstance(v, list):
# Lists used to fall through to the scalar branch below and be
# kept verbatim, blanks and all. Because lists merge by
# *replacement*, saving any unrelated setting then wrote
# [{"token": ""}, ...] straight over the stored list and
# destroyed every credential in it.
pruned = _prune_secret_list(v)
if pruned is not None:
result[k] = pruned
elif v is not None and not (isinstance(v, str) and v.strip() == ''):
result[k] = v
return result
def _prune_secret_list(items: list) -> Optional[list]:
"""Strip blanks from inside a list of secrets, preserving every index.
The rest of the system treats a secrets list as *parallel* to the regular
one -- ``sec[i]`` holds the secret fields of item ``i``, and ``{}`` means
"item i has none" (see ConfigManager._strip_secrets_recursive). So an
emptied dict item stays ``{}``: putting ``None`` there makes that list stop
looking parallel, and the stripper then drops the whole key from the main
config, taking the non-secret fields with it.
A blank *scalar* becomes ``None``, meaning "no update at this index" --
:func:`merge_secrets` substitutes whatever is stored there. Returns
``None`` when nothing in the list carries a real value, so the caller drops
the key and leaves the stored list untouched.
"""
pruned: list = []
has_real_value = False
for item in items:
if isinstance(item, dict):
kept = remove_empty_secrets(item)
pruned.append(kept)
has_real_value = has_real_value or bool(kept)
elif isinstance(item, list):
sub = _prune_secret_list(item)
pruned.append(sub if sub is not None else [])
has_real_value = has_real_value or sub is not None
elif item is not None and not (isinstance(item, str) and item.strip() == ''):
pruned.append(item)
has_real_value = True
else:
pruned.append(None)
return pruned if has_real_value else None
def merge_secrets(stored: Any, incoming: Any) -> Any:
"""Merge submitted secrets over stored ones, element-wise inside lists.
``deep_merge`` replaces a list wholesale. For secrets that is destructive:
an incoming list that carries a real value for one entry and ``None`` for
the rest would drop the stored credentials of every other entry. Here a
list merges by index, and ``None`` means "keep what is stored".
Entries are matched by *position*, which is what the config form gives us
-- there is no schema-declared identity to key on, and it is the same
contract ConfigManager._strip_secrets_recursive already relies on. The
incoming list's length wins, so deleting an item deletes its secrets;
an item the client left blank keeps whatever is stored at that index.
"""
if isinstance(stored, dict) and isinstance(incoming, dict):
merged = dict(stored)
for key, value in incoming.items():
merged[key] = (merge_secrets(stored[key], value)
if key in stored else value)
return merged
if isinstance(stored, list) and isinstance(incoming, list):
# The incoming list sets the length -- the regular config's list is
# authoritative about how many items exist, and this one runs parallel
# to it. Removing an entry must therefore remove its secrets too.
merged_list = []
for index, item in enumerate(incoming):
stored_item = stored[index] if index < len(stored) else None
merged_list.append(stored_item if item is None
else merge_secrets(stored_item, item))
return merged_list
if incoming is None:
return stored
return incoming
def strip_masked_values(secrets: Dict[str, Any]) -> Dict[str, Any]:
"""Remove values a client echoed back rather than changed.
-85
View File
@@ -1,85 +0,0 @@
"""Non-finite JSON numbers must be rejected, not raise.
json.loads accepts Infinity/-Infinity/NaN by default (they are not valid JSON,
but Python's parser emits them) and Flask's get_json passes them straight
through. int(float('inf')) raises OverflowError, which is neither ValueError
nor TypeError -- so validation blocks that carefully caught those let it
through and Flask turned it into a 500.
The damage was not the status code. /config/dim-schedule answered with
CONFIG_SAVE_FAILED and suggested "Check file permissions on config directory"
and "Check available disk space" for what was actually an invalid number.
NaN already returned 400 (int(nan) raises ValueError), which is why this only
showed up for the infinities.
"""
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).parent.parent))
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
#: (route, field) that returned 500 before OverflowError was caught. Both
#: infinity signs are exercised: int() raises OverflowError for either, but
#: only one of them was in the original report, and a guard that special-cased
#: the sign would pass a one-sided test.
NON_FINITE_ROUTES = [
('/api/v3/config/dim-schedule', 'dim_brightness'),
('/api/v3/errors/clear', 'max_age_hours'),
('/api/v3/config/main', 'multiplexing'),
('/api/v3/config/main', 'row_address_type'),
]
NON_FINITE_CASES = [
(route, '{"%s": %s}' % (field, literal))
for route, field in NON_FINITE_ROUTES
for literal in ('Infinity', '-Infinity')
]
@pytest.mark.parametrize("route,body", NON_FINITE_CASES)
def test_infinity_is_a_client_error_not_a_server_error(api_v3_client, route, body):
"""Exactly 400, not merely "some 4xx".
Accepting any 4xx would let a 404 pass, so renaming one of these routes
would leave the test green while testing nothing -- the failure mode this
whole file exists to catch.
"""
response = api_v3_client.post(route, data=body, content_type='application/json')
assert response.status_code == 400, (
f"{route} with {body} answered {response.status_code}; expected 400"
)
@pytest.mark.parametrize("route,body", [
('/api/v3/config/dim-schedule', '{"dim_brightness": NaN}'),
('/api/v3/errors/clear', '{"max_age_hours": NaN}'),
])
def test_nan_is_also_a_client_error(api_v3_client, route, body):
"""int(nan) raises ValueError so this path already worked -- pinned so a
refactor that narrows the except tuple cannot quietly break it."""
response = api_v3_client.post(route, data=body, content_type='application/json')
assert response.status_code == 400
def test_a_valid_number_is_accepted(api_v3_client, api_v3_module, monkeypatch):
"""Prove the widened except did not start swallowing ordinary input.
Asserting "not a 400" would not show that: the mocked save path fails for
any input, so the assertion would hold even if validation had rejected the
value. Give load_config a real dict and stub the atomic save, and the
endpoint reaches its success response -- which only happens if 30 passed
validation.
"""
api_v3_module.api_v3.config_manager.load_config.return_value = {}
monkeypatch.setattr(api_v3_module, '_save_config_atomic',
lambda *a, **k: (True, ''))
response = api_v3_client.post(
'/api/v3/config/dim-schedule',
data='{"dim_brightness": 30}',
content_type='application/json',
)
assert response.status_code == 200, response.get_data(as_text=True)[:200]
-450
View File
@@ -1,450 +0,0 @@
"""A second request for a key already being fetched must join, not duplicate.
request_id embeds a millisecond timestamp and active_requests is keyed by it,
so every submit looked new and nothing compared what was actually being
fetched. On a real board the season-schedule cache_key is requested by both
the Recent and the Upcoming manager: they miss the cache in the same
millisecond and each start a full download and parse of the same payload.
Measured on a running board, 138 background fetches in 24 hours arriving in
pairs at identical timestamps -- half of them redundant.
The cost of a duplicate is a second download, a second JSON parse (the
expensive part on a Pi), and a second parsed copy resident at the same time.
Schedules on that board run from 256KB to 20MB. It also consumes a second of
the three executor slots with identical work, which is what makes two large
parses peak simultaneously.
"""
import threading
import time
from unittest.mock import MagicMock, Mock, patch
import pytest
import requests
from src.background_data_service import BackgroundDataService, FetchStatus
PAYLOAD = {"events": [{"id": f"g{i}"} for i in range(20)]}
@pytest.fixture
def cache():
m = MagicMock()
m.get.return_value = None # always a miss: force the fetch path
m.set.return_value = None
return m
@pytest.fixture
def service(cache):
svc = BackgroundDataService(cache, max_workers=3, request_timeout=5)
yield svc
svc.shutdown(wait=False)
def _resp():
r = Mock()
r.json.return_value = PAYLOAD
r.raise_for_status.return_value = None
return r
def _wait(service, req_id, timeout=5):
deadline = time.time() + timeout
while not service.is_request_complete(req_id) and time.time() < deadline:
time.sleep(0.02)
class _BlockingSession:
"""Holds the first fetch open so a second can be submitted mid-flight."""
def __init__(self):
self.calls = 0
self.release = threading.Event()
self.started = threading.Event()
def get(self, *a, **k):
self.calls += 1
self.started.set()
self.release.wait(timeout=5)
return _resp()
def test_a_second_submit_for_the_same_key_does_not_fetch_twice(service):
session = _BlockingSession()
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="nba_2026",
callback=lambda r: None, max_retries=0)
assert session.started.wait(timeout=5)
second = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="nba_2026",
callback=lambda r: None, max_retries=0)
assert second == first, "the joiner should share the in-flight request id"
session.release.set()
_wait(service, first)
assert session.calls == 1, f"the payload was fetched {session.calls} times"
def test_the_joiner_still_gets_its_callback(service):
session = _BlockingSession()
seen = []
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: seen.append("first"), max_retries=0)
assert session.started.wait(timeout=5)
joined = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: seen.append("second"), max_retries=0)
# Assert the coalescing happened, otherwise this passes trivially:
# two independent requests would each fire their own callback and the
# test would say nothing about the joined path.
assert joined == first
session.release.set()
_wait(service, first)
deadline = time.time() + 5
while len(seen) < 2 and time.time() < deadline:
time.sleep(0.02)
assert sorted(seen) == ["first", "second"], (
f"both submitters must be called back, got {seen}")
def test_one_callback_raising_does_not_silence_the_other(service):
session = _BlockingSession()
seen = []
def boom(result):
raise RuntimeError("consumer blew up")
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=boom, max_retries=0)
assert session.started.wait(timeout=5)
joined = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: seen.append("survivor"), max_retries=0)
# Same reason: without coalescing these are separate requests and
# neither callback can affect the other.
assert joined == first
session.release.set()
_wait(service, first)
deadline = time.time() + 5
while not seen and time.time() < deadline:
time.sleep(0.02)
assert seen == ["survivor"]
def test_different_keys_are_not_coalesced(service):
session = _BlockingSession()
with patch.object(service, "session", session):
a = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/a", cache_key="key_a",
callback=lambda r: None, max_retries=0)
assert session.started.wait(timeout=5)
b = service.submit_fetch_request(
sport="nhl", year=2026, url="https://x/b", cache_key="key_b",
callback=lambda r: None, max_retries=0)
assert a != b, "different cache keys must not share a request"
session.release.set()
_wait(service, a)
_wait(service, b)
assert session.calls == 2
def test_a_later_submit_after_completion_fetches_again(service):
"""Dedupe is for concurrent requests only, not a second cache layer."""
with patch.object(service.session, "get", side_effect=[_resp(), _resp()]) as get:
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: None, max_retries=0)
_wait(service, first)
second = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: None, max_retries=0)
_wait(service, second)
assert first != second
assert get.call_count == 2
def test_cancelling_releases_the_key(service):
"""A cancelled request must not wedge its key against future fetches."""
session = _BlockingSession()
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: None, max_retries=0)
assert session.started.wait(timeout=5)
service.cancel_request(first)
assert "k" not in service._inflight_by_cache_key
session.release.set()
def test_a_stranded_index_entry_cannot_wedge_a_key(service):
"""Defensive: the request is looked up, not trusted from the id alone."""
service._inflight_by_cache_key["ghost"] = "no_such_request"
with patch.object(service.session, "get", return_value=_resp()):
req = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="ghost",
callback=lambda r: None, max_retries=0)
_wait(service, req)
assert service.get_result(req).success is True
def test_the_deduplicated_count_is_reported(service):
session = _BlockingSession()
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: None, max_retries=0)
assert session.started.wait(timeout=5)
service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
max_retries=0)
session.release.set()
_wait(service, first)
assert service.get_statistics().get("deduplicated_requests") == 1
def test_a_cancelled_worker_cannot_overwrite_its_replacement(service, cache):
"""Cancelling frees the key, so a replacement may already own it.
The worker cannot abort an HTTP call in flight, so when the cancelled one
finally returns it must discard its response rather than write it. Without
that, the sequence is: cancel A, submit B for the same key, B fetches and
caches fresh data, A returns and overwrites it with the response nobody
wanted -- and calls A's callbacks too.
"""
slow = _BlockingSession()
stale = {"events": [{"id": "STALE"}]}
slow_resp = Mock()
slow_resp.json.return_value = stale
slow_resp.raise_for_status.return_value = None
def blocked_get(*a, **k):
slow.calls += 1
slow.started.set()
slow.release.wait(timeout=5)
return slow_resp
called = []
with patch.object(service.session, "get", side_effect=blocked_get):
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: called.append("cancelled_one"), max_retries=0)
assert slow.started.wait(timeout=5)
service.cancel_request(first)
assert "k" not in service._inflight_by_cache_key
# The replacement writes the fresh value while the cancelled fetch is held.
fresh = {"events": [{"id": "FRESH"}]}
fresh_resp = Mock()
fresh_resp.json.return_value = fresh
fresh_resp.raise_for_status.return_value = None
with patch.object(service.session, "get", return_value=fresh_resp):
second = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: called.append("replacement"), max_retries=0)
_wait(service, second)
assert cache.set.call_args[0][1] == fresh, "replacement must own the cache"
# Now let the cancelled fetch finish. It must write nothing and call nobody.
# Wait for the worker to actually finish rather than sleeping: a fixed
# sleep is a race under load, and a slow worker would make this pass for
# the wrong reason. A cancelled request is still filed in
# completed_requests, so that is the signal it has run to completion.
writes_before = cache.set.call_count
slow.release.set()
deadline = time.time() + 5
while first not in service.completed_requests and time.time() < deadline:
time.sleep(0.02)
assert first in service.completed_requests, "cancelled worker never finished"
assert cache.set.call_count == writes_before, (
"the cancelled worker wrote to the cache after its replacement")
assert cache.set.call_args[0][1] == fresh, "stale data overwrote fresh"
assert "cancelled_one" not in called, (
"a cancelled request must not deliver callbacks")
def test_request_ids_are_unique_within_a_millisecond(service):
"""request_id was sport_year_milliseconds, which collides.
Two submits inside the same millisecond produced the SAME id, so one
silently replaced the other in active_requests and completed_requests.
Dedupe hands this id back to every joiner as their handle for
get_result(), so uniqueness is now load-bearing rather than incidental.
"""
# Stub the executor rather than the session: this is about what submit
# hands back, and letting 50 workers loose would outlive the patch and
# make real network calls.
with patch.object(service.executor, "submit"):
ids = [
service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s",
cache_key=f"key_{i}", # distinct keys: no dedupe
callback=lambda r: None, max_retries=0)
for i in range(50)
]
assert len(set(ids)) == len(ids), "request ids collided"
# --- cancellation must be terminal ------------------------------------------
#
# Cancelling used to be advisory: three separate paths wrote request.status
# without checking whether the request had already been cancelled, so a cancel
# could be silently undone and the work it was meant to stop went ahead.
URL = "http://example.invalid/scores"
KEY = "sched_nfl_2025"
class _CountingSession:
"""Records whether an HTTP fetch was ever attempted."""
def __init__(self):
self.calls = 0
def get(self, *a, **k):
self.calls += 1
return _resp()
class _BlockingFailingSession(_CountingSession):
"""Holds the fetch open, then fails it.
Cancelling while the worker is parked inside the HTTP call is the only way
to reach the exception handler as a cancelled request. Cancel it before
the call starts and the worker returns at the pre-start branch instead,
which would leave the except path untested.
"""
def __init__(self):
super().__init__()
self.started = threading.Event()
self.release = threading.Event()
def get(self, *a, **k):
self.calls += 1
self.started.set()
self.release.wait(timeout=5)
raise requests.RequestException("connection reset")
def _fill_every_worker_slot(service, slots=3):
"""Occupy the pool so the next submit is queued rather than started.
This is what makes "cancel before the worker runs" deterministic instead
of a race the test would win only sometimes. Returns the gate that
releases the pool.
"""
gate = threading.Event()
for _ in range(slots):
service.executor.submit(gate.wait, 5)
return gate
def test_cancelling_before_the_worker_starts_stops_the_fetch(service, cache):
"""The queued worker must honour a cancel, not overwrite it with IN_PROGRESS.
Between submit and the worker picking the job up, the request sits in the
executor queue. Cancelling there is the cheapest possible cancel -- nothing
has been downloaded yet -- and it was the one that did not work.
"""
gate = _fill_every_worker_slot(service)
session = _CountingSession()
delivered = []
with patch.object(service, 'session', session):
rid = service.submit_fetch_request(
"nfl", 2025, URL, KEY, max_retries=0, callback=delivered.append
)
assert service.cancel_request(rid) is True
gate.set() # let the queued worker run
_wait(service, rid)
assert session.calls == 0, (
"cancelled before it started, yet the worker still downloaded the payload"
)
assert cache.set.call_count == 0, "a cancelled request wrote to the cache"
assert delivered == [], "a cancelled request invoked its callbacks"
def test_a_cancel_during_the_commit_is_refused(service, cache):
"""Once the worker has claimed the commit, cancelling is too late.
The claim and the cancelled-check happen in one critical section, so a
cancel arriving after it cannot retroactively abandon data already on its
way to the cache. Letting it through stranded every joiner: the payload
landed in the cache but the callbacks were suppressed, so a manager that
joined this fetch waited for a call that never came.
"""
gate = _fill_every_worker_slot(service)
late = {}
delivered = []
def cancel_mid_write(key, data, *a, **k):
late['returned'] = service.cancel_request(late['rid'])
cache.set.side_effect = cancel_mid_write
# Read the payload inside the callback. The service releases result.data
# once every callback has been delivered, so inspecting the FetchResult
# afterwards sees the released object, not what the caller was handed.
def record(result):
delivered.append((result.success, result.data))
with patch.object(service, 'session', _CountingSession()):
late['rid'] = service.submit_fetch_request(
"nfl", 2025, URL, KEY, max_retries=0, callback=record
)
gate.set() # only now can the worker reach the commit
_wait(service, late['rid'])
assert late.get('returned') is False, (
"cancelled a request that had already committed"
)
assert cache.set.call_count == 1, "the commit itself was lost"
assert delivered == [(True, PAYLOAD)], (
"data reached the cache but the callbacks were suppressed -- "
"every joined submitter is left waiting forever"
)
def test_a_failure_after_cancelling_stays_cancelled(service, cache):
"""A cancelled request that then errors must not resurface as FAILED.
The except path overwrote CANCELLED with FAILED, and the callback gate in
the finally block only suppresses callbacks for CANCELLED -- so cancelling
a request that was about to time out delivered a spurious error callback.
"""
session = _BlockingFailingSession()
delivered = []
with patch.object(service, 'session', session):
rid = service.submit_fetch_request(
"nfl", 2025, URL, KEY, max_retries=0, callback=delivered.append
)
# Assert the worker is inside the HTTP call before cancelling,
# otherwise this silently degrades into the pre-start case and the
# exception handler is never exercised.
assert session.started.wait(timeout=5)
assert service.cancel_request(rid) is True
session.release.set()
_wait(service, rid)
assert session.calls == 1, "the fetch never started, so nothing could fail"
assert delivered == [], "a cancelled request delivered a failure callback"
assert service.get_request_status(rid) is FetchStatus.CANCELLED, (
"a cancelled request that then errored was reported as FAILED"
)
-280
View File
@@ -1,280 +0,0 @@
"""A delivered fetch payload must not stay resident on the stored result.
BackgroundDataService kept the fetched body on the FetchResult it filed in
`completed_requests`, which is swept only hourly and capped at 500 entries by
count. For status records that is free; for a season schedule it is not. NCAA
football's 2026 schedule is 946 games, and on a 1GB Pi 3B+ the parsed payload
measured ~90MB -- a tenth of the board's memory, pinned for an hour after the
consumer had already been handed it.
The cache-hit path was the worse of the two. It runs once per update interval
per sport, mints a fresh request_id each time, and hands back whatever the
cache returns -- so a memory-tier miss (the tier is capped at 150 entries)
re-parses the payload from disk into a genuinely new object. Those accumulate
as separate copies rather than shared references, which is the staircase seen
in the field: RSS stepping up ~90MB per sport as seasons loaded and never
coming back down.
Releasing is safe because the payload is written to the cache under the
request's cache_key before the result is built, and that is where consumers
read it from -- the callback is handed the object directly and the plugins use
it only in passing before reading the cache back.
Requests submitted *without* a callback keep their payload: polling
get_result() is then the only way to collect it, so releasing would break that
contract.
And the release must happen after EVERY callback, not after each one. Callers
that joined an in-flight fetch share a single FetchResult, so releasing per
delivery strips the payload out from under everyone still queued -- see
TestJoinersAllGetTheData.
"""
import threading
import time
import pytest
from unittest.mock import MagicMock, Mock, patch
from src.background_data_service import BackgroundDataService
PAYLOAD = {"events": [{"id": f"g{i}"} for i in range(50)]}
@pytest.fixture
def cache():
m = MagicMock()
m.get.return_value = None
m.set.return_value = None
return m
@pytest.fixture
def service(cache):
svc = BackgroundDataService(cache, max_workers=2, request_timeout=5)
yield svc
svc.shutdown(wait=False)
def _wait(service, req_id, timeout=5):
"""Wait for the result to be FILED.
Enough for anything that is true by the time the worker stores the result:
its success flag, its error, the cache write that happened during the
fetch.
"""
deadline = time.time() + timeout
while not service.is_request_complete(req_id) and time.time() < deadline:
time.sleep(0.02)
def _wait_for_release(service, req_id, timeout=5):
"""Wait for the payload to be RELEASED, which is strictly later.
The worker files the result, then runs the callback, then releases. So
is_request_complete() goes true while the callback still has not run --
waiting on it alone leaves a window in which `seen` is empty and the
payload is still resident, and the assertions race the worker. It passes
in practice only because a one-line callback usually beats the 20ms poll.
Release happens after the callback returns, so a released payload also
means the callback has finished: one wait covers both.
"""
deadline = time.time() + timeout
while time.time() < deadline:
result = service.get_result(req_id)
if result is not None and result.data is None:
return
time.sleep(0.02)
raise AssertionError(
f"payload for {req_id} was never released (callback may not have run)")
def _resp():
r = Mock()
r.json.return_value = PAYLOAD
r.raise_for_status.return_value = None
return r
class TestFetchPath:
def test_callback_receives_the_payload_then_it_is_released(self, service, cache):
seen = {}
def callback(result):
# The consumer's one look at the data happens here.
seen['events'] = len(result.data['events'])
with patch.object(service.session, "get", return_value=_resp()):
req_id = service.submit_fetch_request(
sport="ncaa_fb", year=2026, url="https://example.com/s",
cache_key="ncaa_fb_2026", callback=callback, max_retries=0,
)
_wait_for_release(service, req_id)
assert seen['events'] == 50, "callback must still be handed the payload"
stored = service.get_result(req_id)
assert stored is not None
assert stored.success is True
assert stored.data is None, "payload must not stay on the stored result"
def test_nothing_is_lost_the_cache_holds_it(self, service, cache):
with patch.object(service.session, "get", return_value=_resp()):
req_id = service.submit_fetch_request(
sport="ncaa_fb", year=2026, url="https://example.com/s",
cache_key="ncaa_fb_2026", callback=lambda r: None, max_retries=0,
)
_wait(service, req_id)
cache.set.assert_called_once()
key, written = cache.set.call_args[0][:2]
assert key == "ncaa_fb_2026"
assert written == PAYLOAD, "the payload must be persisted before release"
def test_without_a_callback_the_payload_is_kept(self, service, cache):
# Polling get_result() is then the only delivery mechanism.
with patch.object(service.session, "get", return_value=_resp()):
req_id = service.submit_fetch_request(
sport="nfl", year=2026, url="https://example.com/s",
cache_key="nfl_2026", max_retries=0,
)
_wait(service, req_id)
assert service.get_result(req_id).data == PAYLOAD
def test_a_failed_fetch_still_records_its_error(self, service, cache):
with patch.object(service.session, "get", side_effect=Exception("boom")):
req_id = service.submit_fetch_request(
sport="nfl", year=2026, url="https://example.com/s",
cache_key="nfl_2026", callback=lambda r: None, max_retries=0,
)
_wait(service, req_id)
stored = service.get_result(req_id)
assert stored.success is False
assert stored.error is not None
class TestCacheHitPath:
def test_cache_hit_releases_after_the_callback(self, service, cache):
cache.get.return_value = PAYLOAD
seen = {}
req_id = service.submit_fetch_request(
sport="ncaa_fb", year=2026, url="https://example.com/s",
cache_key="ncaa_fb_2026",
callback=lambda r: seen.update(events=len(r.data['events'])),
)
assert seen['events'] == 50
assert service.get_result(req_id).data is None
def test_repeated_cache_hits_do_not_accumulate_payloads(self, service, cache):
# The staircase: one entry per update interval per sport, each one
# potentially a freshly parsed copy after a memory-tier miss.
cache.get.return_value = PAYLOAD
for _ in range(25):
service.submit_fetch_request(
sport="ncaa_fb", year=2026, url="https://example.com/s",
cache_key="ncaa_fb_2026", callback=lambda r: None,
)
retained = [r for r in service.completed_requests.values() if r.data is not None]
assert retained == [], f"{len(retained)} payloads still resident"
def test_cache_hit_without_a_callback_is_unchanged(self, service, cache):
cache.get.return_value = PAYLOAD
req_id = service.submit_fetch_request(
sport="nfl", year=2026, url="https://example.com/s",
cache_key="nfl_2026",
)
assert service.get_result(req_id).data == PAYLOAD
class TestJoinersAllGetTheData:
"""Deduplicated callers share one FetchResult; releasing between them
empties it for the rest.
Not a corner case. A sport's recent, upcoming and live managers all ask for
the same season schedule, so the second and third are joiners on almost
every cycle. Releasing inside the delivery loop handed the payload to
whichever ran first and gave the others `result.data is None`.
The consequence was worse than a quiet degradation, because consumers do
`result.data.get('events')`: they raised AttributeError, the delivery loop
caught it, and the whole failure surfaced as a single
"Error in callback for request ..." line while that manager silently never
received its schedule.
"""
class _BlockingSession:
"""Holds the fetch open so a second submit lands while in flight."""
def __init__(self):
self.release = threading.Event()
self.started = threading.Event()
def get(self, *a, **k):
self.started.set()
self.release.wait(timeout=5)
return _resp()
def test_every_joiner_is_handed_the_payload(self, service):
session = self._BlockingSession()
seen = {}
def record(name):
# Read it the way the sport managers do. `result.data['events']`
# would raise TypeError on None; `.get` raises AttributeError,
# which is the error actually seen in the field.
def cb(result):
seen[name] = result.data.get('events') if result.data else None
return cb
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nhl", year=2026, url="https://x/s", cache_key="nhl_2026",
callback=record("first"), max_retries=0)
assert session.started.wait(timeout=5)
joined = service.submit_fetch_request(
sport="nhl", year=2026, url="https://x/s", cache_key="nhl_2026",
callback=record("second"), max_retries=0)
# Without coalescing these are two independent fetches that each
# own their result, and the test proves nothing about sharing.
assert joined == first, "the joiner should share the in-flight id"
session.release.set()
_wait(service, first)
deadline = time.time() + 5
while len(seen) < 2 and time.time() < deadline:
time.sleep(0.02)
assert set(seen) == {"first", "second"}, f"both must be called, got {seen}"
for name, events in seen.items():
assert events is not None, (
f"{name!r} was handed a released payload: the result was "
f"emptied before every callback had been delivered")
assert len(events) == 50, f"{name!r} got {events!r}"
def test_the_payload_is_still_released_once_they_have_all_had_it(self, service):
"""The memory fix must survive the ordering fix."""
session = self._BlockingSession()
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nhl", year=2026, url="https://x/s", cache_key="nhl_2026",
callback=lambda r: None, max_retries=0)
assert session.started.wait(timeout=5)
service.submit_fetch_request(
sport="nhl", year=2026, url="https://x/s", cache_key="nhl_2026",
callback=lambda r: None, max_retries=0)
session.release.set()
_wait_for_release(service, first)
stored = service.get_result(first)
assert stored is not None and stored.data is None, (
"the payload must still be dropped once every callback has run")
+301
View File
@@ -0,0 +1,301 @@
"""The composer generates Python that the plugin loader imports and executes.
/api/install writes the generated manager.py into plugins_dir and the loader
imports it, so anything the payload can splice into that source runs on the
device. The ast.parse check in _generate_plugin_files rejects only *invalid*
syntax -- an injected `import os` is perfectly valid and passed it.
Two ways in, both confirmed against the code before it was fixed:
metadata.name = a name containing a triple-quote, a newline, then
`import os; PWNED = os.getuid()`, then another triple-quote
-> closes the module docstring; the rest became module-level statements
(spelled out rather than shown literally -- writing the payload into
this docstring closes *this* file's docstring, which is the bug)
element x = '0 or __import__("os").system("id")'
-> f-string interpolated it verbatim: x=0 or __import__("os").system("id")
"""
import ast
import re
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from web_interface.blueprints import composer as C # noqa: E402
BASE_META = {"id": "test-plugin", "name": "Clock", "author": "a",
"version": "1.0.0", "description": "d"}
#: Values that terminate a Python expression and start a new statement.
EXPR_PAYLOADS = [
'0 or __import__("os").system("id")',
'0);import os;os.system("id");(',
'__import__("subprocess").run(["id"])',
"0 if False else exec('x=1')",
"1e999", "nan", "0x41", "0__0",
]
#: Values that close a string literal in the generated source.
LITERAL_PAYLOADS = [
'Clock"""\nimport os; PWNED = os.getuid()\n"""',
"Clock'''\nimport os\n'''",
'Clock" + __import__("os").system("id") + "',
"Clock\\", "Clock\nimport os",
]
def _payload(**over):
# dataModel.configVars is the key _generate_plugin_files reads; "config_vars"
# was never looked at, so anything passed through it tested nothing.
p = {"metadata": dict(BASE_META), "elements": [],
"dataModel": {"configVars": over.pop("config_vars", [])}}
p["metadata"].update(over.pop("metadata", {}))
p.update(over)
return p
def _generated(payload):
return C._generate_plugin_files(payload)["manager.py"]
def _module_level_code(src):
"""Statements at module level that are not the docstring/imports/classes."""
tree = ast.parse(src)
out = []
for node in tree.body:
if isinstance(node, (ast.ClassDef, ast.FunctionDef, ast.ImportFrom)):
continue
if isinstance(node, ast.Expr) and isinstance(node.value, ast.Constant):
continue # the docstring
out.append(ast.unparse(node))
return out
@pytest.mark.parametrize("payload", LITERAL_PAYLOADS)
def test_a_name_that_breaks_out_of_a_literal_is_refused(payload):
with pytest.raises(C.ComposerInputError):
_generated(_payload(metadata={"name": payload}))
#: Types with a drawing branch in manager.py.j2. An injection test using any
#: other type proves nothing: _preprocess_elements drops it, so its values
#: never reach the generated source and every assertion passes trivially.
#: This test previously used "line", which has never had a branch.
RENDERED_GEOMETRY_CASES = [
("rectangle", {"x": 0, "y": 0, "width": 10, "height": 8}),
("arc", {"x": 0, "y": 0, "width": 24, "height": 24}),
("ellipse", {"x": 0, "y": 0, "width": 24, "height": 12}),
("rounded_rectangle", {"x": 0, "y": 0, "width": 24, "height": 10}),
("gauge", {"x": 0, "y": 0, "width": 32, "height": 32}),
]
@pytest.mark.parametrize("etype,base", RENDERED_GEOMETRY_CASES)
@pytest.mark.parametrize("evil", EXPR_PAYLOADS)
@pytest.mark.parametrize("field", ["x", "y", "width", "height"])
def test_a_non_numeric_geometry_value_cannot_reach_the_source(etype, base, evil, field):
"""width/height were interpolated raw into the generated source.
p['x2_expr'] = f"({x_expr}) + {w}" with w straight off the payload, so a
rectangle with width='0 or __import__("os").system("id")' produced
[0, 0, (0) + 0 or __import__("os").system("id"), (0) + 8],
in a manager.py that /api/install writes to disk and the loader imports.
"""
el = {"type": etype, "id": "e1", **base}
el[field] = evil
src = _generated(_payload(elements=[el]))
assert "__import__" not in src, f"{etype}.{field}={evil!r} reached the generated source"
assert "os.system" not in src
assert not _module_level_code(src), \
f"{etype}.{field}={evil!r} produced module-level statements: {_module_level_code(src)}"
def test_every_injection_case_uses_a_type_that_actually_renders():
"""Guards against the whole suite quietly going vacuous again.
An element type with no template branch is dropped before generation, so
an injection test written against one asserts nothing and still passes.
"""
used = {etype for etype, _ in RENDERED_GEOMETRY_CASES}
missing = used - set(C._RENDERABLE_ELEMENT_TYPES)
assert not missing, f"injection tests use non-rendering types: {sorted(missing)}"
@pytest.mark.parametrize("evil", EXPR_PAYLOADS)
@pytest.mark.parametrize("channel", ["r", "g", "b"])
def test_a_non_numeric_colour_channel_cannot_reach_the_source(evil, channel):
el = {"type": "text", "id": "t1", "x": 0, "y": 0, "text": "hi",
"font": "press_start", "r": 255, "g": 255, "b": 255}
el[channel] = evil
src = _generated(_payload(elements=[el]))
assert "__import__" not in src and "os.system" not in src
assert not _module_level_code(src)
def test_colour_channels_are_clamped_to_a_byte():
el = {"type": "text", "id": "t1", "x": 0, "y": 0, "text": "hi",
"font": "press_start", "r": 99999, "g": -5, "b": 128}
src = _generated(_payload(elements=[el]))
assert "(255, 0, 128)" in src, "channels were not clamped to 0-255"
def test_the_generated_module_still_has_no_top_level_statements():
"""The clean case: a normal payload produces only imports and a class."""
el = {"type": "text", "id": "t1", "x": 4, "y": 4, "text": "hi",
"font": "press_start", "r": 1, "g": 2, "b": 3}
src = _generated(_payload(elements=[el]))
assert not _module_level_code(src)
assert "(1, 2, 3)" in src
# --- config variable keys ---------------------------------------------------
def _with_key(key):
return {"metadata": dict(BASE_META), "elements": [],
"dataModel": {"configVars": [{"key": key, "type": "string",
"default": "x", "label": "L"}]}}
@pytest.mark.parametrize("key", ["class", "def", "import", "None", "True",
"lambda", "pass", "match", "case"])
def test_a_keyword_config_key_is_named_in_the_error(key):
"""ast.parse already rejected these, but as an unhelpful line number.
"Generated code has a syntax error: invalid syntax (line 17)" tells the
user nothing about which field to fix.
"""
with pytest.raises(C.ComposerInputError) as exc:
_generated(_with_key(key))
assert key in str(exc.value) and "keyword" in str(exc.value).lower()
@pytest.mark.parametrize("key", ["config", "logger", "display_manager",
"cache_manager", "plugin_id", "enabled",
"self", "update", "display"])
def test_a_reserved_attribute_config_key_is_refused(key):
"""These generate *valid* Python that silently clobbers plugin state.
The worst is `config`: the assignment lands right after super().__init__(),
so `self.config = config.get("config", "x")` replaces the plugin's config
dict with a string and every later self.config.get(...) fails at runtime.
"""
with pytest.raises(C.ComposerInputError) as exc:
_generated(_with_key(key))
assert key in str(exc.value) and "reserved" in str(exc.value).lower()
@pytest.mark.parametrize("key", ["brightness", "my_var", "_private", "x1",
"update_interval_seconds"])
def test_ordinary_config_keys_are_still_accepted(key):
src = _generated(_with_key(key))
assert f"self.{key} = config.get(" in src
def test_the_generated_config_assignment_does_not_precede_super_init():
"""Guards the reasoning behind the reserved list, not just the list."""
src = _generated(_with_key("brightness"))
body = src.splitlines()
super_at = next(i for i, line in enumerate(body) if "super().__init__(" in line)
assign_at = next(i for i, line in enumerate(body)
if "self.brightness = config.get(" in line)
assert assign_at > super_at, (
"config vars are assigned before super().__init__(); the reserved-name "
"list assumes they land after it")
# --- optional keys ----------------------------------------------------------
@pytest.mark.parametrize("el_type,missing", [
("text", "text"), ("text", "text2"), ("clock", "format"),
])
def test_an_element_missing_an_optional_key_does_not_500(el_type, missing):
"""`p` is a copy of the raw element, so an absent key stays absent.
The defaults were applied to locals only, so manager.py.j2 rendered
`{{ el.text | tojson }}` over a jinja2.Undefined and tojson raised
TypeError -- which no handler catches, making a missing key a 500 rather
than a validation error or a sensible default.
"""
el = {"type": el_type, "id": "e1", "x": 0, "y": 0, "font": "press_start"}
src = _generated(_payload(elements=[el]))
ast.parse(src) # must still be valid Python
assert "Undefined" not in src
def test_a_clock_without_a_format_uses_the_documented_default():
el = {"type": "clock", "id": "c1", "x": 0, "y": 0, "font": "press_start"}
src = _generated(_payload(elements=[el]))
assert '"%H:%M"' in src, "the %H:%M default did not reach the generated source"
#: (element type, channel key, base element) for colour channels that were
#: interpolated raw rather than through _rgb_expr/_safe_int. Prefixed channels
#: (emptyR/G/B, labelR/G/B) were the ones the original r/g/b test never reached.
RAW_COLOUR_CASES = [
("progress_bar", "r", {"x": 0, "y": 0}),
("progress_bar", "g", {"x": 0, "y": 0}),
("pips", "b", {"x": 0, "y": 0}),
("pips", "emptyR", {"x": 0, "y": 0}),
("pips", "emptyG", {"x": 0, "y": 0}),
("sparkline", "r", {"x": 0, "y": 0}),
("gauge", "labelR", {"x": 0, "y": 0, "width": 32, "height": 32}),
("gauge", "labelB", {"x": 0, "y": 0, "width": 32, "height": 32}),
]
@pytest.mark.parametrize("etype,channel,base", RAW_COLOUR_CASES)
@pytest.mark.parametrize("evil", EXPR_PAYLOADS)
def test_a_prefixed_colour_channel_cannot_reach_the_source(etype, channel, base, evil):
"""Five tuples were built with f"({el.get('r', 100)}, ...)" -- no coercion.
The pre-existing colour test only covered r/g/b on a text element, so the
prefixed channels and the four other types were never exercised.
"""
el = {"type": etype, "id": "e1", **base}
el[channel] = evil
src = _generated(_payload(elements=[el]))
assert "__import__" not in src, f"{etype}.{channel}={evil!r} reached the source"
assert "os.system" not in src
assert not _module_level_code(src)
@pytest.mark.parametrize("value", [float("inf"), float("-inf"), float("nan")])
@pytest.mark.parametrize("field", ["x", "y", "width", "height"])
def test_a_non_finite_dimension_does_not_escape_as_an_unhandled_error(field, value):
"""json.loads accepts Infinity/NaN and Flask passes them through, so a
payload can hand _safe_int a non-finite float. int(inf) raises
OverflowError -- neither ValueError nor ComposerInputError -- so it escaped
both handlers and surfaced as a 500 with a traceback instead of a 422."""
el = {"type": "rectangle", "id": "r1", "x": 0, "y": 0, "width": 10, "height": 8}
el[field] = value
src = _generated(_payload(elements=[el])) # must not raise
# A non-finite value must be replaced by the default, not spelled into the
# source. Word-boundary match: "info" in self.logger.info contains "inf".
assert not re.search(r"\b(inf|nan|Infinity|NaN)\b", src), \
f"{field}={value!r} leaked a non-finite literal into the source"
assert not _module_level_code(src)
@pytest.mark.parametrize("bad_id", [
'x = __import__("os").system("id") #',
"x\nimport os\n_y",
"x[0]",
"",
"a" * 200,
])
def test_a_marquee_id_cannot_become_code(bad_id):
"""data_key is spliced UNQUOTED into variable names
(_{{ data_key }}_text = ...), so a non-identifier id landed in the source
as code. ast.parse caught it, but the caller then got an opaque
"Generated code has a syntax error" rather than being told the id is bad."""
el = {"type": "marquee", "id": bad_id, "x": 0, "y": 0, "text": "hi"}
src = _generated(_payload(elements=[el])) # must not raise
assert "__import__(" not in src
assert "os.system(" not in src
assert not _module_level_code(src)
+80
View File
@@ -0,0 +1,80 @@
"""An element the template cannot draw must not produce an empty `if` block.
manager.py.j2 wraps each element in `if width >= N:` (breakpoint) and/or
`if int(time.time() * 2) % 2:` (blink), and the body comes from the per-type
branches. A type with no branch contributed nothing, so the wrapper opened a
block with no statements in it. ast.parse in _generate_plugin_files then
failed and the caller was told only:
Generated code has a syntax error: expected an indented block after
'if' statement on line 49
which names a line of generated source the user never sees. Confirmed against
the code before the fix with a `group` element carrying minWidth.
Two defences, both covered here: _preprocess_elements drops types the template
has no branch for, and the template emits a `pass` fallback so a type added to
the canvas before its branch exists degrades to a no-op instead of a broken
plugin.
"""
import re
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from web_interface.blueprints import composer as C # noqa: E402
TEMPLATE = (Path(__file__).resolve().parent.parent
/ "web_interface/templates/v3/composer/manager.py.j2")
BASE_META = {"id": "test-plugin", "name": "Clock", "author": "a",
"version": "1.0.0", "description": "d"}
def generate(element):
return C._generate_plugin_files({
"metadata": BASE_META,
"elements": [element],
"dataModel": {"configVars": []},
})
@pytest.mark.parametrize("wrapper", [
{"minWidth": 64}, # breakpoint block
{"blink": True}, # blink block
{"minWidth": 64, "blink": True}, # both, nested
])
@pytest.mark.parametrize("etype", ["group", "widget_9000", "section"])
def test_undrawable_element_does_not_break_generation(etype, wrapper):
element = {"type": etype, "x": 0, "y": 0, "color": "#ffffff", **wrapper}
files = generate(element) # must not raise ComposerInputError
assert "manager.py" in files
def test_drawable_element_still_renders_inside_a_breakpoint():
files = generate({"type": "text", "text": "hi", "x": 0, "y": 0,
"minWidth": 64, "color": "#ffffff"})
src = files["manager.py"]
assert "if width >= 64:" in src
assert "draw_text" in src
def test_renderable_types_match_the_template_branches():
"""The constant and the template must agree.
A type listed in the constant with no branch emits an empty block (the bug
above); a type with a branch but missing from the constant is silently
dropped from every generated plugin. Neither is visible without this check.
"""
branches = set(re.findall(r"el\.type == '([a-z_]+)'", TEMPLATE.read_text()))
assert branches == set(C._RENDERABLE_ELEMENT_TYPES)
def test_template_closes_the_branch_chain_with_a_fallback():
"""Belt and braces: even if the constant drifts, no empty block escapes."""
text = TEMPLATE.read_text()
assert "{% else %}" in text
assert "pass # element type" in text
+147
View File
@@ -0,0 +1,147 @@
"""Structural checks on the composer's JavaScript.
There is no JS test runner in this repo, but three defects here are structural
enough to assert on the parse tree, and each was a real bug:
- Stroke widths inside _drawElement were left in canvas pixels while the
geometry around them scaled by SCALE, so at SCALE>1 every outline rendered
thinner than one LED pixel and the preview stopped matching the panel.
- The `line` branch drew raw el.x0/el.y0/el.x1/el.y1, ignoring the anchor that
every other element type honours, so setting xAnchor moved everything except
lines -- and getBoundingBox had the same omission, leaving the hit box behind.
- Four methods mutated exactly what _snapshot() serialises (metadata,
currentPreset) without calling it. _snapshot is the only caller of
_debouncedAutosave, so those changes were lost on reload and could not be
undone.
"""
import re
from pathlib import Path
import pytest
tree_sitter = pytest.importorskip("tree_sitter")
tree_sitter_javascript = pytest.importorskip("tree_sitter_javascript")
JS_DIR = Path(__file__).resolve().parent.parent / "web_interface/static/v3/js/composer"
CANVAS = JS_DIR / "composer-canvas.js"
APP = JS_DIR / "composer-app.js"
def _function_source(path: Path, name: str) -> str:
"""Return the source of a top-level function declaration by name."""
src = path.read_bytes()
lang = tree_sitter.Language(tree_sitter_javascript.language())
tree = tree_sitter.Parser(lang).parse(src)
found = []
def walk(node):
if node.type == "function_declaration":
ident = node.child_by_field_name("name")
if ident is not None and src[ident.start_byte:ident.end_byte].decode() == name:
found.append(src[node.start_byte:node.end_byte].decode())
for c in node.children:
walk(c)
walk(tree.root_node)
assert found, f"{name} not found in {path.name}"
return found[0]
def _method_source(path: Path, name: str) -> str:
"""Return the source of a top-level object method by name."""
src = path.read_bytes()
lang = tree_sitter.Language(tree_sitter_javascript.language())
tree = tree_sitter.Parser(lang).parse(src)
found = []
def walk(node):
if node.type == "method_definition":
ident = node.child_by_field_name("name")
if ident is not None and src[ident.start_byte:ident.end_byte].decode() == name:
found.append(src[node.start_byte:node.end_byte].decode())
for c in node.children:
walk(c)
walk(tree.root_node)
assert found, f"{name} not found in {path.name}"
return found[0]
def test_both_files_parse():
lang = tree_sitter.Language(tree_sitter_javascript.language())
parser = tree_sitter.Parser(lang)
for path in (CANVAS, APP):
tree = parser.parse(path.read_bytes())
errors = []
def walk(node):
if node.type == "ERROR" or node.is_missing:
errors.append(node.start_point[0] + 1)
for c in node.children:
walk(c)
walk(tree.root_node)
assert not errors, f"{path.name} has parse errors at lines {errors}"
def test_element_strokes_scale_with_scale():
"""No bare `ctx.lineWidth = 1` inside _drawElement.
Selection handles and the grid are drawn in canvas pixels deliberately and
live in other functions, so this is scoped to the element drawing routine.
"""
body = _function_source(CANVAS, "_drawElement")
offenders = re.findall(r"ctx\.lineWidth\s*=\s*1\s*;", body)
assert not offenders, f"{len(offenders)} unscaled stroke width(s) in _drawElement"
def test_line_branch_applies_the_anchor_offset():
"""Scoped to _drawElement.
getBoundingBox has its own `case 'line': {` and appears first in the file,
so searching the whole text found *that* branch -- this assertion passed
with the draw branch's anchor offset removed. Verified: stripping it and
re-running gave 11/11 green.
"""
body = _function_source(CANVAS, "_drawElement")
line_branch = body[body.index("case 'line': {"):]
line_branch = line_branch[:line_branch.index("case 'divider'")]
assert "ax - el.x0" in line_branch and "ay - el.y0" in line_branch, \
"line drawing ignores xAnchor/yAnchor"
assert "moveTo(el.x0 * s" not in line_branch, \
"line still drawn from unanchored endpoints"
def test_line_bounding_box_applies_the_anchor_offset():
"""The companion to the above: scoped to getBoundingBox specifically, so
the two tests cannot both be satisfied by the same branch."""
body = _function_source(CANVAS, "getBoundingBox")
box = body[body.index("case 'line'"):]
box = box[:box.index("case 'divider'")]
assert "ax - el.x0" in box, "line bounding box ignores the anchor"
@pytest.mark.parametrize("method", [
"onBgColorChange", # mutates metadata.bgColor
"setCustomSize", # mutates currentPreset / MATRIX_W / MATRIX_H
"changePreset", # mutates currentPreset / MATRIX_W / MATRIX_H
"applyPresetLabel", # same, for sizes not in DISPLAY_PRESETS
"onColorChange", # the one that was already fixed — keeps it fixed
])
def test_state_mutations_take_a_snapshot(method):
body = _method_source(APP, method)
assert "_snapshot()" in body, \
f"{method} changes snapshotted state without calling _snapshot()"
assert "isDirty = true" in body, f"{method} does not mark the design dirty"
@pytest.mark.parametrize("method", ["changePreset", "applyPresetLabel"])
def test_restore_path_stays_snapshot_free(method):
"""_applyState and loadTemplate call these with {silent: true} while
restoring; snapshotting there would push restore steps onto the undo stack
and re-autosave the state just loaded."""
body = _method_source(APP, method)
assert "opts.silent" in body, f"{method} lost its silent guard"
snap = body.index("_snapshot()")
guard = body.index("!opts.silent")
assert guard < snap, f"{method} snapshots outside the !opts.silent guard"
+225
View File
@@ -0,0 +1,225 @@
"""A composer plugin id must never resolve outside the plugins directory.
CodeQL reported sixteen high-severity py/path-injection alerts against
web_interface/blueprints/composer.py: a request-supplied plugin_id reaching
Path(plugins_dir) / plugin_id, which is then created, written to, deleted
(shutil.rmtree) and read back.
The id was already validated by an anchored regex, so every traversal payload
was in fact rejected. What was missing was the guarantee living *with* the path
building rather than in a regex several hundred lines away -- loosen that regex
later and the traversal opens silently, with nothing at the filesystem boundary
to catch it. _plugin_dir() closes that, and is the form static analysis can see.
"""
import sys
from pathlib import Path
from unittest.mock import MagicMock
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from web_interface.blueprints import composer as C # noqa: E402
#: Anything that has ever been used to climb out of a directory.
TRAVERSAL = [
"../../etc/passwd", "..", ".", "a/../../etc", "good/../../..",
"/etc/passwd", "//etc/passwd", "a\\..\\..", "a%2f..%2f..",
"....//....//etc", "a/./../../etc", "~", "~root",
"plugin/../../../../../../etc/shadow",
]
#: Rejected for shape, not traversal -- but rejected all the same.
MALFORMED = ["", "A-upper", "1-leading-digit", "-leading-dash", "has_underscore",
"has space", "has.dot", "a" * 64, "plugin\n", "plugin\n../../etc",
"\n", "plug\x00in"]
@pytest.fixture
def plugins_dir(tmp_path, monkeypatch):
base = tmp_path / "plugin-repos"
base.mkdir()
monkeypatch.setattr(C.composer_bp, "plugins_dir", str(base), raising=False)
return base
@pytest.mark.parametrize("payload", TRAVERSAL)
def test_traversal_payloads_are_refused(plugins_dir, payload):
assert C._plugin_dir(payload) is None
@pytest.mark.parametrize("payload", MALFORMED)
def test_malformed_ids_are_refused(plugins_dir, payload):
assert C._plugin_dir(payload) is None
@pytest.mark.parametrize("payload", ["a", "my-plugin", "x9", "a" * 63])
def test_valid_ids_resolve_inside_the_base(plugins_dir, payload):
resolved = C._plugin_dir(payload)
assert resolved is not None, f"{payload!r} was rejected but is valid"
assert resolved.parent == plugins_dir.resolve(), (
f"{payload!r} resolved to {resolved}, outside {plugins_dir}")
def test_no_payload_can_escape_even_if_the_regex_is_loosened(plugins_dir, monkeypatch):
"""The containment check must stand on its own.
This is the whole point of resolving at the filesystem boundary: if the id
pattern is ever relaxed, traversal must still be impossible. Replace the
regex with one that permits slashes and dots, then re-run the payloads.
"""
import re
monkeypatch.setattr(C, "_PLUGIN_ID_RE", re.compile(r"\A[\w./\\~-]+\Z"))
import os
escaped = []
base = os.path.realpath(str(plugins_dir))
for payload in TRAVERSAL:
resolved = C._plugin_dir(payload)
if resolved is None:
continue
real = os.path.realpath(str(resolved))
if real != base and os.path.commonpath([base, real]) != base:
escaped.append((payload, real))
assert not escaped, f"these escaped the base with a loosened regex: {escaped}"
def test_a_sibling_directory_with_a_shared_prefix_is_not_inside(tmp_path, monkeypatch):
"""commonpath, not startswith.
"/x/plugins-evil" starts with "/x/plugins" but is a different directory, so
a prefix test would accept it.
"""
base = tmp_path / "plugins"
base.mkdir()
(tmp_path / "plugins-evil").mkdir()
monkeypatch.setattr(C.composer_bp, "plugins_dir", str(base), raising=False)
import re
# Neutralise the two layers in front so this exercises the containment
# check itself; otherwise secure_filename rejects the payload first and a
# startswith regression would go unnoticed here.
monkeypatch.setattr(C, "_PLUGIN_ID_RE", re.compile(r"\A[\w./\\~-]+\Z"))
monkeypatch.setattr(C, "secure_filename", lambda v: v)
assert C._plugin_dir("../plugins-evil") is None
def test_containment_still_holds_if_the_sanitiser_is_defeated(plugins_dir, monkeypatch):
"""Each layer is tested on its own, not just the stack.
secure_filename's equality guard rejects every traversal payload before the
containment check sees it, so removing containment does not fail the other
tests -- which would make it look load-bearing when it is not. Neutralise
the regex *and* the sanitiser, and the realpath/commonpath check must still
refuse everything on its own.
"""
import re
monkeypatch.setattr(C, "_PLUGIN_ID_RE", re.compile(r"\A[\w./\\~-]+\Z"))
monkeypatch.setattr(C, "secure_filename", lambda v: v)
import os
base = os.path.realpath(str(plugins_dir))
escaped = []
for payload in TRAVERSAL:
resolved = C._plugin_dir(payload)
if resolved is None:
continue
real = os.path.realpath(str(resolved))
# Inside the base is fine -- "...." and "~" are ordinary directory
# names on Linux, so they are not escapes. What must never happen is
# landing outside the base, or on the base itself: install() rmtrees
# its target, so the plugins root resolving to a "plugin" would wipe
# every installed plugin.
if real == base or os.path.commonpath([base, real]) != base:
escaped.append((payload, real))
assert not escaped, f"containment alone let these through: {escaped}"
def test_secure_filename_never_rewrites_an_accepted_id(plugins_dir):
"""The sanitiser must be a no-op on everything the regex accepts.
If secure_filename ever altered an accepted id, _plugin_dir would resolve
to a *different* plugin's directory than the caller asked for -- a silent
redirect, which is worse than a refusal. The guard turns that into a
refusal; this proves the guard never has to fire in practice.
"""
import random
from werkzeug.utils import secure_filename
random.seed(1)
alphabet = "abcdefghijklmnopqrstuvwxyz0123456789-"
altered = []
for _ in range(2000):
n = random.randint(1, 63)
cand = random.choice("abcdefghijklmnopqrstuvwxyz") + "".join(
random.choice(alphabet) for _ in range(n - 1))
if C._PLUGIN_ID_RE.match(cand) and secure_filename(cand) != cand:
altered.append((cand, secure_filename(cand)))
assert not altered, f"secure_filename rewrote accepted ids: {altered[:5]}"
def test_a_trailing_newline_is_not_a_valid_id():
r"""Python's `$` also matches before a trailing newline, so the original
`^...$` accepted "myplugin\n" and would have created a directory whose
name ends in one. \Z does not."""
assert C._PLUGIN_ID_RE.match("myplugin") is not None
assert C._PLUGIN_ID_RE.match("myplugin\n") is None
# --- font serving -----------------------------------------------------------
FONT_TRAVERSAL = [
"../../../etc/passwd", "../config/config.json", "..%2f..%2fetc%2fpasswd",
"PressStart2P-Regular.ttf/../../../etc/passwd", "/etc/passwd", "",
"PressStart2P-Regular.TTF", # case differs -> not the allowlisted name
"PressStart2P-Regular.ttf ", # trailing space
]
def test_serve_font_refuses_a_file_that_exists_but_is_not_allowlisted(monkeypatch, tmp_path):
"""The allowlist must be what refuses it, not a missing file.
Asserting 404 on traversal payloads proves nothing here: Flask's router
will not match a path segment containing '/', and everything else 404s
simply because no such file exists. Put a real, readable file next to the
fonts and confirm it is still refused -- that is the allowlist working.
"""
fonts = tmp_path / "assets" / "fonts"
fonts.mkdir(parents=True)
(fonts / "id_rsa.ttf").write_bytes(b"PRIVATE KEY")
monkeypatch.setattr(C.composer_bp, "project_root", str(tmp_path), raising=False)
app = __import__("flask").Flask(__name__)
app.register_blueprint(C.composer_bp)
with app.test_client() as client:
resp = client.get("/api/fonts/id_rsa.ttf")
assert resp.status_code == 404, (
"a readable non-allowlisted file was served; the allowlist is not gating")
assert b"PRIVATE KEY" not in resp.data
@pytest.mark.parametrize("payload", FONT_TRAVERSAL)
def test_serve_font_refuses_anything_not_allowlisted(payload, monkeypatch, tmp_path):
"""The name reaching the filesystem must come from the allowlist constant.
_ALLOWED_FONTS gates this endpoint, so nothing here was ever exploitable.
Building the path from the matched constant rather than the request value
is what makes that provable -- and it is why CodeQL reported two
high-severity py/path-injection alerts on an endpoint that was already
safe.
"""
monkeypatch.setattr(C.composer_bp, "project_root", str(tmp_path), raising=False)
app = C.composer_bp.name and __import__("flask").Flask(__name__)
app.register_blueprint(C.composer_bp)
with app.test_client() as client:
resp = client.get(f"/api/fonts/{payload}")
assert resp.status_code in (404, 405, 308), (
f"{payload!r} was not refused (status {resp.status_code})")
def test_serve_font_still_serves_each_allowlisted_font(monkeypatch, tmp_path):
fonts = tmp_path / "assets" / "fonts"
fonts.mkdir(parents=True)
monkeypatch.setattr(C.composer_bp, "project_root", str(tmp_path), raising=False)
app = __import__("flask").Flask(__name__)
app.register_blueprint(C.composer_bp)
for name in C._ALLOWED_FONTS:
(fonts / name).write_bytes(b"\x00\x01ttf")
with app.test_client() as client:
resp = client.get(f"/api/fonts/{name}")
assert resp.status_code == 200, f"{name} should be served, got {resp.status_code}"
@@ -6,7 +6,6 @@ These tests cover the reconcile path that loads/unloads plugins and rebuilds
the dispatch maps on the main thread when the enabled set changes.
"""
import copy
from unittest.mock import MagicMock
@@ -254,182 +253,3 @@ class TestEnabledSetChanged:
{"a": {"enabled": True, "duration": 30}},
{"a": {"enabled": True, "duration": 45}},
) is False
class TestEnabledPluginNotRunning:
"""A plugin that fails validate_config() is enabled but absent, and the
config edit that fixes it is nested inside the plugin's own section -- so
the top-level ``enabled`` comparison never sees it. These cover the second
gate that queues a reconcile in that case.
"""
def test_nested_edit_is_invisible_to_the_enabled_set_check(self, test_display_controller):
"""The original gate: proves why a second one is needed."""
controller = test_display_controller
old = {"hockey-scoreboard": {"enabled": True, "nhl": {"enabled": False}}}
new = {"hockey-scoreboard": {"enabled": True, "nhl": {"enabled": True}}}
# Enabling a league changes no top-level flag.
assert controller._enabled_set_changed(old, new) is False
def test_queues_reconcile_when_enabled_plugin_is_absent(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {} # failed to load
cfg = {"hockey-scoreboard": {"enabled": True, "nhl": {"enabled": True}}}
assert controller._enabled_plugin_not_running(cfg) is True
def test_quiet_when_every_enabled_plugin_is_running(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {"hockey-scoreboard": ["nhl"]}
cfg = {"hockey-scoreboard": {"enabled": True}}
assert controller._enabled_plugin_not_running(cfg) is False
def test_disabled_plugin_does_not_queue(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {}
cfg = {"hockey-scoreboard": {"enabled": False}}
assert controller._enabled_plugin_not_running(cfg) is False
def test_non_plugin_sections_do_not_queue(self, test_display_controller):
"""``schedule``/``display`` carry their own ``enabled`` and are never
in plugin_display_modes -- without the manifest check they would queue
a reconcile, and therefore a filesystem scan, on every config save."""
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {"hockey-scoreboard": ["nhl"]}
cfg = {
"hockey-scoreboard": {"enabled": True},
"schedule": {"enabled": True},
"display": {"enabled": True},
}
assert controller._enabled_plugin_not_running(cfg) is False
def test_non_dict_section_is_ignored(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {}
assert controller._enabled_plugin_not_running({"hockey-scoreboard": "nonsense"}) is False
def test_no_plugin_manager_is_quiet(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager = None
assert controller._enabled_plugin_not_running({"x": {"enabled": True}}) is False
class TestReconcileQueuedThroughSubscriber:
"""End-to-end through the real config-change subscriber, not the helper.
Without the second gate this is the four-day-outage path: the plugin is
enabled, absent, and the save that enables its league sets no flag.
"""
@staticmethod
def _subscriber(controller):
subs = controller.config_service._subscribers['*']
for cb in subs:
if getattr(cb, '__name__', '') == '_controller_config_change':
return cb
raise AssertionError(f"controller subscriber not found among {subs}")
@staticmethod
def _configs(controller, plugin_section_old, plugin_section_new):
"""Build two full configs differing only inside the plugin section --
the subscriber refreshes its cache from these, so they must be real."""
base = copy.deepcopy(controller.config)
old = copy.deepcopy(base)
new = copy.deepcopy(base)
old["hockey-scoreboard"] = plugin_section_old
new["hockey-scoreboard"] = plugin_section_new
return old, new
def test_nested_edit_queues_reconcile_for_absent_plugin(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {} # validate_config() said False
controller._pending_plugin_reconcile = False
old, new = self._configs(
controller,
{"enabled": True, "nhl": {"enabled": False}},
{"enabled": True, "nhl": {"enabled": True}},
)
# The original gate is blind to this edit ...
assert controller._enabled_set_changed(old, new) is False
self._subscriber(controller)(old, new)
# ... but the reconcile is queued anyway.
assert controller._pending_plugin_reconcile is True
def test_steady_state_does_not_queue_reconcile(self, test_display_controller):
"""Everything enabled is running: an unrelated edit must not queue a
reconcile, or every config save drags a filesystem scan onto the
render thread."""
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {"hockey-scoreboard": ["nhl"]}
controller._pending_plugin_reconcile = False
old, new = self._configs(
controller,
{"enabled": True, "scroll_speed": 1},
{"enabled": True, "scroll_speed": 2},
)
self._subscriber(controller)(old, new)
assert controller._pending_plugin_reconcile is False
class TestPendingReconcileNotLost:
"""A config change arriving *during* reconcile must not be discarded.
The flag used to be cleared after a successful reconcile. Reconcile has
already read its config by then, so that clear erased a request it never
served and the newest config never reconciled -- the same "my save did
nothing" symptom this path exists to prevent.
"""
def test_request_arriving_during_reconcile_survives(self, test_display_controller):
controller = test_display_controller
controller._pending_plugin_reconcile = True
def reconcile_and_race():
# The watcher thread queues another change while we are mid-flight.
with controller._reconcile_flag_lock:
controller._pending_plugin_reconcile = True
return True
controller._reconcile_enabled_plugins = reconcile_and_race
controller._service_pending_reconcile()
assert controller._pending_plugin_reconcile is True, \
"a config change landing during reconcile was discarded"
def test_flag_cleared_on_a_quiet_success(self, test_display_controller):
controller = test_display_controller
controller._pending_plugin_reconcile = True
controller._reconcile_enabled_plugins = lambda: True
controller._service_pending_reconcile()
assert controller._pending_plugin_reconcile is False
def test_retryable_failure_rearms(self, test_display_controller):
controller = test_display_controller
controller._pending_plugin_reconcile = True
controller._reconcile_enabled_plugins = lambda: False
controller._service_pending_reconcile()
assert controller._pending_plugin_reconcile is True
def test_no_reconcile_when_nothing_pending(self, test_display_controller):
controller = test_display_controller
controller._pending_plugin_reconcile = False
calls = []
controller._reconcile_enabled_plugins = lambda: calls.append(1) or True
controller._service_pending_reconcile()
assert calls == []
+2 -34
View File
@@ -13,7 +13,6 @@ need root and mutate the system, so they are exercised manually instead.
"""
import subprocess
import tempfile
from pathlib import Path
import pytest
@@ -32,16 +31,6 @@ def run_lib(snippet: str, env: dict | None = None) -> subprocess.CompletedProces
)
def _fstype_of(path: object) -> str:
"""Filesystem type backing ``path``, via the same tool the helper uses."""
result = subprocess.run(
["findmnt", "-no", "FSTYPE", "--target", str(path)],
capture_output=True, text=True,
env={"PATH": "/usr/bin:/bin:/usr/sbin:/sbin"},
)
return result.stdout.strip()
def call(fn: str, *args: object, env: dict | None = None) -> str:
joined = " ".join(str(a) for a in args)
result = run_lib(f"{fn} {joined}", env=env)
@@ -206,29 +195,8 @@ class TestOomDetection:
class TestDiskBackedTmpdir:
def test_returns_nothing_when_tmpdir_is_already_disk_backed(self, tmp_path):
# Do not assume tmp_path is disk-backed. Debian 13 -- the platform this
# helper exists for -- mounts /tmp as tmpfs, and pytest puts tmp_path
# under /tmp, so this asserted against a *memory*-backed directory and
# failed on the target platform while the helper behaved exactly as
# designed. Search for a directory whose backing store is really disk.
scratch = None
disk_backed = None
for candidate in (tmp_path, Path("/var/tmp"), LIB.parent):
if _fstype_of(candidate) not in ("tmpfs", "ramfs", ""):
if candidate is tmp_path:
disk_backed = candidate
else:
scratch = Path(tempfile.mkdtemp(dir=str(candidate)))
disk_backed = scratch
break
if disk_backed is None:
pytest.skip("no disk-backed directory available to test against")
try:
assert call("lm_disk_backed_tmpdir",
env={"TMPDIR": str(disk_backed)}) == ""
finally:
if scratch is not None:
scratch.rmdir()
# tmp_path is on the regular filesystem, so the default must be kept.
assert call("lm_disk_backed_tmpdir", env={"TMPDIR": str(tmp_path)}) == ""
def test_redirects_away_from_a_memory_backed_tmpdir(self):
# Debian 13 mounts /tmp as tmpfs, which would otherwise hold the whole
+2 -8
View File
@@ -89,17 +89,11 @@ class TestContextualFormatter:
assert "hello" in out
def test_location_toggle(self):
# Assert on the whole "module.func:lineno" token, not a bare ":42".
# The formatted line starts with an HH:MM:SS timestamp, so a bare
# ":{lineno}" also matches the clock whenever the minute or second
# happens to equal the line number -- about 3% of runs, which is a
# flaky failure with nothing wrong.
record = make_record()
location = f"{record.module}.{record.funcName}:{record.lineno}"
with_loc = ContextualFormatter(include_location=True).format(record)
without = ContextualFormatter(include_location=False).format(record)
assert location in with_loc
assert location not in without
assert f":{record.lineno}" in with_loc
assert f":{record.lineno}" not in without
def test_record_not_mutated_no_double_prefix(self):
# Regression: a record is formatted once PER HANDLER. The formatter
+1 -225
View File
@@ -8,27 +8,11 @@ ensure_logo_directory, and the download_missing_logo function path
"""
import os
import time
import pytest
from pathlib import Path
from unittest.mock import patch, Mock, MagicMock
from PIL import Image
from PIL.PngImagePlugin import PngInfo
from src.logo_downloader import (
PLACEHOLDER_BG,
PLACEHOLDER_MARKER,
PLACEHOLDER_RETRY_SECONDS,
PLACEHOLDER_SIZE,
LogoDownloader,
download_missing_logo,
is_placeholder_logo,
placeholder_age_seconds,
refresh_placeholder_timestamp,
should_attempt_download,
)
from src.logo_downloader import LogoDownloader
# ---------------------------------------------------------------------------
@@ -143,211 +127,3 @@ class TestEnsureLogoDirectory:
with patch("builtins.open", side_effect=mock_open):
result = downloader.ensure_logo_directory(test_dir)
assert result is False
# ---------------------------------------------------------------------------
# Placeholder detection and retry
#
# A failed download used to be cached as a placeholder wearing the real logo's
# filename, and download_missing_logo returned early on "the file exists". One
# transient failure therefore pinned a team to a grey box permanently.
# ---------------------------------------------------------------------------
class TestPlaceholderLogos:
def _placeholder(self, tmp_path, abbrev="COLL"):
downloader = LogoDownloader()
assert downloader.create_placeholder_logo(abbrev, str(tmp_path)) is True
return tmp_path / f"{abbrev}.png"
def test_generated_placeholder_is_recognised(self, tmp_path):
assert is_placeholder_logo(self._placeholder(tmp_path)) is True
def test_real_logo_is_not_a_placeholder(self, tmp_path):
real = tmp_path / "REAL.png"
Image.new("RGBA", (500, 500), (12, 34, 56, 255)).save(real)
assert is_placeholder_logo(real) is False
def test_legacy_unmarked_placeholder_is_recognised(self, tmp_path):
"""Placeholders written before the marker existed must still be caught.
They are already sitting on users' disks; if they were not recognised
those teams would stay grey boxes forever even after this fix.
"""
legacy = tmp_path / "LEGACY.png"
Image.new("RGBA", PLACEHOLDER_SIZE, PLACEHOLDER_BG).save(legacy)
assert is_placeholder_logo(legacy) is True
def test_same_size_but_different_colour_is_not_a_placeholder(self, tmp_path):
real = tmp_path / "SMALL.png"
Image.new("RGBA", PLACEHOLDER_SIZE, (10, 200, 10, 255)).save(real)
assert is_placeholder_logo(real) is False
def test_missing_file_is_not_a_placeholder(self, tmp_path):
assert is_placeholder_logo(tmp_path / "nope.png") is False
def test_existing_real_logo_short_circuits_without_downloading(self, tmp_path):
real = tmp_path / "REAL.png"
Image.new("RGBA", (500, 500), (1, 2, 3, 255)).save(real)
with patch.object(LogoDownloader, "download_logo") as download:
assert download_missing_logo(
"afl", "1", "REAL", real, logo_url="http://example/x.png") is True
download.assert_not_called()
def _age_placeholder(self, path, seconds):
"""Rewrite a placeholder's marker so it reads as `seconds` old."""
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time() - seconds))
with Image.open(path) as img:
img.copy().save(path, "PNG", pnginfo=metadata)
def test_stale_placeholder_triggers_a_retry(self, tmp_path):
path = self._placeholder(tmp_path)
self._age_placeholder(path, PLACEHOLDER_RETRY_SECONDS + 60)
assert placeholder_age_seconds(path) > PLACEHOLDER_RETRY_SECONDS
with patch.object(LogoDownloader, "download_logo", return_value=True) as download:
assert download_missing_logo(
"afl", "1", "COLL", path,
logo_url="http://example/coll.png") is True
download.assert_called_once()
def test_placeholder_age_survives_an_mtime_touch(self, tmp_path):
"""The age comes from the stamp, not the filesystem.
Anything that rewrites file times -- a backup restore, an rsync, a
permissions fix script -- would otherwise reset the retry clock.
"""
path = self._placeholder(tmp_path)
self._age_placeholder(path, PLACEHOLDER_RETRY_SECONDS + 60)
now = time.time()
os.utime(path, (now, now))
assert placeholder_age_seconds(path) > PLACEHOLDER_RETRY_SECONDS
def test_fresh_placeholder_does_not_retry(self, tmp_path):
"""Rate limiting: a placeholder written seconds ago must not re-download.
Without this the fix would trade a permanent grey box for an ESPN
request on every frame.
"""
path = self._placeholder(tmp_path)
with patch.object(LogoDownloader, "download_logo") as download:
assert download_missing_logo(
"afl", "1", "COLL", path,
logo_url="http://example/coll.png") is True
download.assert_not_called()
class TestDownloadEligibility:
"""One rule, shared by every download site.
The two bulk loops and the single-logo path each had their own idea of what
counted as "already have it", which is how one of them ended up retrying
fresh placeholders and the other skipping stale ones forever.
"""
def _placeholder(self, tmp_path, abbrev="COLL"):
assert LogoDownloader().create_placeholder_logo(abbrev, str(tmp_path))
return tmp_path / f"{abbrev}.png"
def _age(self, path, seconds):
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time() - seconds))
with Image.open(path) as img:
img.copy().save(path, "PNG", pnginfo=metadata)
def test_missing_file_is_eligible(self, tmp_path):
assert should_attempt_download(tmp_path / "nope.png") is True
def test_real_logo_is_not_eligible(self, tmp_path):
real = tmp_path / "REAL.png"
Image.new("RGBA", (500, 500), (1, 2, 3, 255)).save(real)
assert should_attempt_download(real) is False
def test_force_download_beats_a_real_logo(self, tmp_path):
real = tmp_path / "REAL.png"
Image.new("RGBA", (500, 500), (1, 2, 3, 255)).save(real)
assert should_attempt_download(real, force_download=True) is True
def test_fresh_placeholder_is_not_eligible(self, tmp_path):
assert should_attempt_download(self._placeholder(tmp_path)) is False
def test_stale_placeholder_is_eligible(self, tmp_path):
path = self._placeholder(tmp_path)
self._age(path, PLACEHOLDER_RETRY_SECONDS + 60)
assert should_attempt_download(path) is True
def test_league_bulk_loop_skips_a_fresh_placeholder(self, tmp_path):
"""A bulk pass honours the same back-off as everything else."""
self._placeholder(tmp_path, "AAA")
downloader = LogoDownloader()
teams = [{"abbreviation": "AAA", "display_name": "A", "logo_url": "http://x/a.png"}]
with patch.object(LogoDownloader, "get_logo_directory", return_value=str(tmp_path)):
with patch.object(LogoDownloader, "fetch_teams_data", return_value={"sports": [{}]}):
with patch.object(LogoDownloader, "extract_teams_from_data", return_value=teams):
with patch.object(LogoDownloader, "download_logo") as download:
downloader.download_missing_logos_for_league("nfl")
download.assert_not_called()
def test_league_bulk_loop_retries_a_stale_placeholder(self, tmp_path):
path = self._placeholder(tmp_path, "AAA")
self._age(path, PLACEHOLDER_RETRY_SECONDS + 60)
downloader = LogoDownloader()
teams = [{"abbreviation": "AAA", "display_name": "A", "logo_url": "http://x/a.png"}]
with patch.object(LogoDownloader, "get_logo_directory", return_value=str(tmp_path)):
with patch.object(LogoDownloader, "fetch_teams_data", return_value={"sports": [{}]}):
with patch.object(LogoDownloader, "extract_teams_from_data", return_value=teams):
with patch.object(LogoDownloader, "download_logo", return_value=True) as download:
downloader.download_missing_logos_for_league("nfl")
download.assert_called_once()
def test_ncaa_bulk_loop_retries_a_stale_placeholder(self, tmp_path):
"""This loop skipped placeholders forever; it now shares the rule."""
path = self._placeholder(tmp_path, "AAA")
self._age(path, PLACEHOLDER_RETRY_SECONDS + 60)
downloader = LogoDownloader()
teams = [{"abbreviation": "AAA", "display_name": "A",
"logo_url": "http://x/a.png", "category": "FBS",
"conference": "SEC"}]
with patch.object(LogoDownloader, "get_logo_directory", return_value=str(tmp_path)):
with patch.object(LogoDownloader, "fetch_teams_data", return_value={"sports": [{}]}):
with patch.object(LogoDownloader, "extract_teams_from_data", return_value=teams):
with patch.object(LogoDownloader, "download_logo", return_value=True) as download:
downloader.download_all_ncaa_football_logos()
download.assert_called_once()
def test_ncaa_bulk_loop_skips_a_fresh_placeholder(self, tmp_path):
self._placeholder(tmp_path, "AAA")
downloader = LogoDownloader()
teams = [{"abbreviation": "AAA", "display_name": "A",
"logo_url": "http://x/a.png", "category": "FBS",
"conference": "SEC"}]
with patch.object(LogoDownloader, "get_logo_directory", return_value=str(tmp_path)):
with patch.object(LogoDownloader, "fetch_teams_data", return_value={"sports": [{}]}):
with patch.object(LogoDownloader, "extract_teams_from_data", return_value=teams):
with patch.object(LogoDownloader, "download_logo") as download:
downloader.download_all_ncaa_football_logos()
download.assert_not_called()
class TestRefreshPlaceholderTimestamp:
def test_restarts_the_back_off(self, tmp_path):
assert LogoDownloader().create_placeholder_logo("COLL", str(tmp_path))
path = tmp_path / "COLL.png"
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time() - (PLACEHOLDER_RETRY_SECONDS + 60)))
with Image.open(path) as img:
img.copy().save(path, "PNG", pnginfo=metadata)
assert should_attempt_download(path) is True
assert refresh_placeholder_timestamp(path) is True
assert should_attempt_download(path) is False
def test_refuses_to_touch_a_real_logo(self, tmp_path):
real = tmp_path / "REAL.png"
Image.new("RGBA", (500, 500), (1, 2, 3, 255)).save(real)
before = real.read_bytes()
assert refresh_placeholder_timestamp(real) is False
assert real.read_bytes() == before
def test_missing_file_is_not_an_error(self, tmp_path):
assert refresh_placeholder_timestamp(tmp_path / "nope.png") is False
-73
View File
@@ -421,76 +421,3 @@ class TestSessionConfiguration:
def test_user_agent_and_accept_headers(self, helper):
assert helper.session.headers["User-Agent"] == "LEDMatrix-Common/1.0"
assert helper.session.headers["Accept"] == "image/*"
class TestStalePlaceholderHandling:
"""load_logo_with_download must not be fooled by a cached placeholder.
A placeholder wears the real logo's filename, so both the file cache and
the in-memory cache can hold one and look like a hit.
"""
def _placeholder(self, tmp_path, abbrev="COLL"):
from src.logo_downloader import LogoDownloader
assert LogoDownloader().create_placeholder_logo(abbrev, str(tmp_path))
return tmp_path / f"{abbrev}.png"
def _make_stale(self, path):
import time
from PIL.PngImagePlugin import PngInfo
from src.logo_downloader import PLACEHOLDER_MARKER, PLACEHOLDER_RETRY_SECONDS
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time() - (PLACEHOLDER_RETRY_SECONDS + 60)))
with Image.open(path) as img:
img.copy().save(path, "PNG", pnginfo=metadata)
def test_fresh_placeholder_is_served_without_a_download(self, helper, tmp_path):
path = self._placeholder(tmp_path)
with patch.object(LogoHelper, "_download_logo") as download:
assert helper.load_logo_with_download("COLL", path, "http://x/c.png") is not None
download.assert_not_called()
def test_stale_placeholder_triggers_a_download(self, helper, tmp_path):
path = self._placeholder(tmp_path)
self._make_stale(path)
with patch.object(LogoHelper, "_download_logo") as download:
helper.load_logo_with_download("COLL", path, "http://x/c.png")
download.assert_called_once()
def test_replacement_logo_is_not_masked_by_the_cached_placeholder(self, helper, tmp_path):
"""The bug this guards: load_logo answers from cache before the disk.
Without invalidation the freshly downloaded logo would not appear until
the process restarted.
"""
path = self._placeholder(tmp_path)
first = helper.load_logo_with_download("COLL", path, "http://x/c.png")
assert first is not None
self._make_stale(path)
def fake_download(_self, _url, file_path):
Image.new("RGB", (500, 500), (7, 8, 9)).save(file_path, format="PNG")
with patch.object(LogoHelper, "_download_logo", fake_download):
second = helper.load_logo_with_download("COLL", path, "http://x/c.png")
assert second is not None
from src.logo_downloader import is_placeholder_logo
assert is_placeholder_logo(path) is False
assert second.getpixel((0, 0))[:3] == (7, 8, 9)
def test_failed_retry_restarts_the_back_off(self, helper, tmp_path):
"""Otherwise a stale placeholder means a download attempt per call."""
from src.logo_downloader import should_attempt_download
path = self._placeholder(tmp_path)
self._make_stale(path)
assert should_attempt_download(path) is True
def boom(_self, _url, _file_path):
raise OSError("network down")
with patch.object(LogoHelper, "_download_logo", boom):
helper.load_logo_with_download("COLL", path, "http://x/c.png")
assert should_attempt_download(path) is False
+3 -298
View File
@@ -19,7 +19,6 @@ The rules being pinned here, in priority order:
"""
import json
import subprocess
from pathlib import Path
from unittest.mock import MagicMock
@@ -147,18 +146,9 @@ def store(tmp_path, monkeypatch):
class TestInstallGate:
"""`install_plugin` is the chokepoint for every route that re-downloads:
`_reinstall_with_rollback` calls it, so a refused update restores the
version the user already had.
It is not the *only* route in, and saying so here once is cheaper than
rediscovering it. `update_plugin` also has a git branch that pulls in
place and never re-downloads; that one is gated separately by
`_gate_pulled_commit` and pinned in `TestGitPullGate` below. A third
route, `install_from_url` (sideloading from a URL), is gated in that
function and pinned in `TestSideloadGate`. All three refuse on the same
rule.
"""
"""`install_plugin` is the chokepoint: `_reinstall_with_rollback` calls it,
so gating there covers updates too, and a refused update restores the
version the user already had."""
def _install_with_manifest(self, store, manifest, core_version, monkeypatch):
mgr, plugins_dir = store
@@ -218,291 +208,6 @@ class TestInstallGate:
assert (path / "manifest.json").exists()
class TestSideloadGate:
"""`install_from_url` never looked at the core version.
Sideloading is an explicit act rather than an automatic store update, so
the argument for gating it is different: not "the user did not choose
this", but that the floor states the plugin *cannot run here*. Letting it
through produces the same silent PluginState.ERROR at load that the store
gate exists to prevent, and the user who typed the URL is no better placed
to diagnose it than one who pressed Update.
"""
def _sideload(self, store, manifest, core_version, monkeypatch):
mgr, plugins_dir = store
plugin_id = manifest["id"]
def fake_clone(repo_url, target_path, branches=None):
target_path.mkdir(parents=True, exist_ok=True)
(target_path / "manifest.json").write_text(
json.dumps(manifest), encoding="utf-8")
(target_path / "manager.py").write_text(
"class P: pass\n", encoding="utf-8")
return "main"
monkeypatch.setattr(mgr, "_install_via_git", fake_clone)
monkeypatch.setattr(mgr, "_install_dependencies", lambda *a, **k: True)
import src
monkeypatch.setattr(src, "__version__", core_version)
return mgr.install_from_url("https://example.invalid/plugin"), \
plugins_dir / plugin_id
def test_refuses_a_plugin_that_needs_a_newer_core(self, store, monkeypatch):
manifest = {
"id": "sideload-newer", "name": "Sideload", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "9.9.9",
}
result, path = self._sideload(store, manifest, "3.2.0", monkeypatch)
assert result["success"] is False
assert "9.9.9" in result["error"], result["error"]
assert not path.exists(), (
"a refused sideload must not leave the plugin installed")
def test_allows_a_compatible_plugin(self, store, monkeypatch):
"""The guard against over-refusing: a gate that blocks everything
passes the test above and breaks sideloading entirely."""
manifest = {
"id": "sideload-fine", "name": "Sideload", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "3.0.0",
}
result, path = self._sideload(store, manifest, "3.2.0", monkeypatch)
assert result["success"] is True, result.get("error")
assert (path / "manifest.json").exists()
def test_untrustworthy_core_does_not_block_a_2_0_0_floor(
self, store, monkeypatch):
"""Same rule as the other two routes: a v3.1.0 release reports 1.0.0,
and nearly every published manifest floors at 2.0.0."""
manifest = {
"id": "sideload-floored", "name": "Sideload", "class_name": "P",
"display_modes": ["a"], "versions": [{"ledmatrix_min": "2.0.0"}],
}
result, _ = self._sideload(store, manifest, "1.0.0", monkeypatch)
assert result["success"] is True, result.get("error")
# --------------------------------------------------------------------------
# The git-pull update path — the one route that does not re-download
# --------------------------------------------------------------------------
def _git_available() -> bool:
# OSError, not just a non-zero exit: with no git on PATH subprocess raises
# FileNotFoundError, and this runs at import time -- before skipif can act,
# so the whole module would error out instead of skipping.
try:
return subprocess.run(
['git', '--version'], capture_output=True).returncode == 0
except OSError:
return False
_HAS_GIT = _git_available()
def _git(*args, cwd):
return subprocess.run(['git', *args], cwd=str(cwd),
capture_output=True, text=True, check=True)
@pytest.mark.skipif(not _HAS_GIT, reason='git not available')
class TestGitPullGate:
"""A plugin installed as a git checkout updates by pulling in place, so it
never passes through `install_plugin` and was never gated.
Real git repositories rather than mocks, because the claim under test is
that `git reset --hard` puts the checkout back — a mock of git would only
prove the call was made, which is the easy half.
Only the handful of registry entries with no `plugin_path` reach this path
in practice; monorepo plugins install as archives and update through
`_reinstall_with_rollback`. It is gated anyway because the sunset rule in
`docs/plugin-development/08-shared-sports-code.md` states the core enforces
the floor "at install/update time", and a precondition that is documented
but not true is worse than one that is merely missing.
"""
@pytest.fixture
def checkout(self, tmp_path, monkeypatch):
"""An origin repo holding a plugin, and a clone of it installed as
`plugin-repos/gitplug`, with the store pointed at it."""
from src.plugin_system.store_manager import PluginStoreManager
origin = tmp_path / 'origin'
origin.mkdir()
_git('init', '--initial-branch=main', '--bare', cwd=origin)
seed = tmp_path / 'seed'
_git('clone', str(origin), str(seed), cwd=tmp_path)
_git('config', 'user.email', 'test@example.com', cwd=seed)
_git('config', 'user.name', 'Test', cwd=seed)
(seed / 'manifest.json').write_text(json.dumps({
"id": "gitplug", "name": "Git Plug", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "2.0.0",
}), encoding='utf-8')
(seed / 'manager.py').write_text("class P: pass\n", encoding='utf-8')
_git('add', '-A', cwd=seed)
_git('commit', '-m', 'initial', cwd=seed)
_git('push', '-u', 'origin', 'main', cwd=seed)
plugins_dir = tmp_path / 'plugin-repos'
plugins_dir.mkdir()
work = plugins_dir / 'gitplug'
_git('clone', str(origin), str(work), cwd=tmp_path)
_git('config', 'user.email', 'test@example.com', cwd=work)
_git('config', 'user.name', 'Test', cwd=work)
mgr = PluginStoreManager(plugins_dir=str(plugins_dir))
mgr.logger = MagicMock()
# No registry entry, so update_plugin takes the plain-pull branch
# rather than the remote-mismatch or already-current shortcuts.
monkeypatch.setattr(mgr, 'fetch_registry', lambda *a, **k: None)
monkeypatch.setattr(mgr, 'get_plugin_info', lambda *a, **k: None)
monkeypatch.setattr(mgr, '_install_dependencies', lambda *a, **k: True)
return mgr, seed, work
def _push(self, seed, manifest_text):
(seed / 'manifest.json').write_text(manifest_text, encoding='utf-8')
_git('add', '-A', cwd=seed)
_git('commit', '-m', 'update', cwd=seed)
_git('push', 'origin', 'main', cwd=seed)
def _head(self, work):
return _git('rev-parse', 'HEAD', cwd=work).stdout.strip()
def test_refuses_a_pulled_commit_that_needs_a_newer_core(
self, checkout, monkeypatch):
mgr, seed, work = checkout
before = self._head(work)
self._push(seed, json.dumps({
"id": "gitplug", "name": "Git Plug", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "9.9.9",
}))
import src
monkeypatch.setattr(src, '__version__', '3.2.0')
assert mgr.update_plugin('gitplug') is False
assert self._head(work) == before, (
"a refused update must leave the checkout on the commit it was "
"already running, not on the one it cannot load")
floor = json.loads((work / 'manifest.json').read_text())
assert floor['min_ledmatrix_version'] == '2.0.0', (
"the reset must restore the working tree, not just the ref")
def test_allows_a_pulled_commit_the_core_can_run(
self, checkout, monkeypatch):
"""The guard against over-refusing. A gate that refuses everything
passes the test above and breaks every update."""
mgr, seed, work = checkout
before = self._head(work)
self._push(seed, json.dumps({
"id": "gitplug", "name": "Git Plug", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "3.0.0",
}))
import src
monkeypatch.setattr(src, '__version__', '3.2.0')
assert mgr.update_plugin('gitplug') is True
assert self._head(work) != before
def test_an_unreadable_manifest_after_pull_is_not_a_refusal(
self, checkout, monkeypatch):
"""Rule 1 of this file — refuse only on evidence. A manifest that
will not parse declares no floor, so it is not evidence of anything."""
mgr, seed, work = checkout
self._push(seed, '{ this is not json')
import src
monkeypatch.setattr(src, '__version__', '3.2.0')
assert mgr.update_plugin('gitplug') is True
def test_an_untrustworthy_core_does_not_block_a_2_0_0_floor_on_pull(
self, checkout, monkeypatch):
"""The same regression guard as
`test_untrustworthy_core_does_not_block_installs`, because this path
now shares that rule: a v3.1.0 release reports 1.0.0, and nearly every
published manifest floors at 2.0.0."""
mgr, seed, work = checkout
before = self._head(work)
self._push(seed, json.dumps({
"id": "gitplug", "name": "Git Plug", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "2.0.0",
"description": "changed",
}))
import src
monkeypatch.setattr(src, '__version__', '1.0.0')
assert mgr.update_plugin('gitplug') is True
assert self._head(work) != before
def test_a_failed_stash_stops_the_update_before_pulling(
self, checkout, monkeypatch):
"""Uncommitted work is not collateral for the gate.
The gate's only rollback is `git reset --hard`, which discards
uncommitted tracked edits. update_plugin stashes them first -- but when
that stash fails it used to pull anyway, and a pull touching different
files succeeds on a dirty tree. Refuse, refuse the rollback's rollback,
and the user's edits go with it.
"""
mgr, seed, work = checkout
before = self._head(work)
(work / 'manager.py').write_text(
"class P:\n MINE = 'do not lose this'\n", encoding='utf-8')
self._push(seed, json.dumps({
"id": "gitplug", "name": "Git Plug", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "9.9.9",
}))
real_run = subprocess.run
def fail_stash(cmd, *a, **k):
if isinstance(cmd, list) and 'stash' in cmd:
return subprocess.CompletedProcess(cmd, 1, '', 'stash boom')
return real_run(cmd, *a, **k)
import src
monkeypatch.setattr(src, '__version__', '3.2.0')
monkeypatch.setattr(
'src.plugin_system.store_manager.subprocess.run', fail_stash)
assert mgr.update_plugin('gitplug') is False
assert self._head(work) == before, "must not pull what it cannot undo"
assert 'do not lose this' in (work / 'manager.py').read_text(), (
"the local edit the stash failed to save must still be there")
def test_rollback_reports_the_recovery_command_when_git_fails(
self, checkout, monkeypatch):
"""If the reset itself fails the user is left on a version that cannot
load, so the log line has to carry the command that fixes it —
it is the only thing standing between them and a manual reinstall."""
mgr, seed, work = checkout
self._push(seed, json.dumps({
"id": "gitplug", "name": "Git Plug", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "9.9.9",
}))
real_run = subprocess.run
def fail_reset(cmd, *a, **k):
if isinstance(cmd, list) and 'reset' in cmd:
return subprocess.CompletedProcess(cmd, 1, '', 'reset boom')
return real_run(cmd, *a, **k)
import src
monkeypatch.setattr(src, '__version__', '3.2.0')
monkeypatch.setattr(
'src.plugin_system.store_manager.subprocess.run', fail_reset)
assert mgr.update_plugin('gitplug') is False
logged = ' '.join(str(c) for c in mgr.logger.error.call_args_list)
assert 'reset --hard' in logged, logged
class TestLoaderAndStoreAgree:
"""Both read the same manifests; a disagreement means one of them is
lying to the user."""
@@ -1,63 +0,0 @@
"""Tests for PluginManager.discovered_plugin_ids().
The config-watcher thread needs the set of discovered plugin ids while the
render thread may be rebuilding plugin_manifests. Iterating that dict directly
can observe a half-populated mapping or raise "dictionary changed size during
iteration", so the accessor snapshots it under the discovery lock.
"""
import tempfile
import threading
from pathlib import Path
import pytest
from src.plugin_system.plugin_manager import PluginManager
@pytest.fixture
def pm():
with tempfile.TemporaryDirectory() as tmp:
yield PluginManager(plugins_dir=str(Path(tmp) / "plugins"))
def test_returns_the_discovered_ids(pm):
pm.plugin_manifests = {"clock-simple": {}, "hockey-scoreboard": {}}
assert pm.discovered_plugin_ids() == {"clock-simple", "hockey-scoreboard"}
def test_empty_when_nothing_discovered(pm):
pm.plugin_manifests = {}
assert pm.discovered_plugin_ids() == set()
def test_is_a_snapshot_not_a_live_view(pm):
"""The caller iterates the result on another thread; it must not alias
the mapping discovery is still writing to."""
pm.plugin_manifests = {"clock-simple": {}}
snapshot = pm.discovered_plugin_ids()
pm.plugin_manifests["hockey-scoreboard"] = {}
assert snapshot == {"clock-simple"}
def test_takes_the_discovery_lock(pm):
"""Guards against the lock being dropped in a later refactor: with the
lock held by another thread the call must block rather than read."""
pm.plugin_manifests = {"clock-simple": {}}
finished = threading.Event()
def call():
pm.discovered_plugin_ids()
finished.set()
pm._discovery_lock.acquire()
try:
# RLock is reentrant per-thread, so use a *different* thread to prove
# the accessor actually waits on it.
t = threading.Thread(target=call, daemon=True)
t.start()
assert not finished.wait(timeout=0.3), "accessor did not take the discovery lock"
finally:
pm._discovery_lock.release()
t.join(timeout=2)
assert finished.is_set()
-166
View File
@@ -1,166 +0,0 @@
"""Plugin state history must not grow without bound.
`PluginStateManager` recorded every state transition in a per-plugin list and
never trimmed it. The only code that removed entries was `clear_state()`, called
solely from `PluginManager.unload_plugin()`, so a plugin that stays loaded --
i.e. normal operation -- never released a single entry.
The list is written on the hot scheduling path. Every update cycle appends
twice: `_reserve_for_update()` sets RUNNING and `_finish()` sets ENABLED back
again. At the default 60-second update interval that is 2,880 entries per
plugin per day, and nothing ever reads the entries -- `get_state_info()` only
takes their `len()`. It is pure dead weight.
Measured against the unpatched class, ten plugins on a 60s interval retain
864,010 transitions after thirty simulated days, for 231 MB of heap. On a 1 GB
Pi that is fatal on its own, and the failure is not a clean OOM: once
MemAvailable falls far enough, fork() starts returning ENOMEM, so sshd accepts
connections and closes them before its banner while the kernel still answers
pings. The board looks like a hardware fault and needs a power cycle.
These tests pin the cap, the retention order, and the one piece of behaviour the
cap must not change: `state_history_count` is surfaced through the web API, so
it has to keep reporting the lifetime total rather than plateauing at the cap.
"""
import os
import sys
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.plugin_system.plugin_state import ( # noqa: E402
MAX_STATE_HISTORY_PER_PLUGIN,
PluginState,
PluginStateManager,
)
def _cycle_updates(manager, plugin_id, cycles):
"""Drive the real scheduling path: RUNNING on reserve, ENABLED on finish."""
for _ in range(cycles):
manager.set_state(plugin_id, PluginState.RUNNING)
manager.set_state(plugin_id, PluginState.ENABLED)
def test_state_history_is_capped():
"""A day of updates must not retain a day of transitions."""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
# One simulated day at the default 60s update interval.
_cycle_updates(manager, "clock", 1440)
history = manager.get_state_history("clock")
assert len(history) <= MAX_STATE_HISTORY_PER_PLUGIN, (
f"history grew to {len(history)} entries; it is never trimmed"
)
def test_state_history_keeps_the_most_recent_transitions():
"""Trimming drops the oldest entries, not the newest."""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
_cycle_updates(manager, "clock", MAX_STATE_HISTORY_PER_PLUGIN)
history = manager.get_state_history("clock")
# The scheduling cycle ends on ENABLED, so the newest entry is the
# RUNNING -> ENABLED half of the last cycle.
assert history[-1]["from"] == PluginState.RUNNING.value
assert history[-1]["to"] == PluginState.ENABLED.value
# And the very first ENABLED transition has aged out.
assert history[0]["from"] != PluginState.UNLOADED.value
def test_state_history_count_reports_lifetime_total():
"""The count exposed through the API must not plateau at the cap.
`get_state_info()['state_history_count']` is surfaced by the web UI. Capping
the retained list must not turn it into "entries we happen to still hold".
"""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
total = 1
cycles = MAX_STATE_HISTORY_PER_PLUGIN * 2
_cycle_updates(manager, "clock", cycles)
total += cycles * 2
info = manager.get_state_info("clock")
assert info["state_history_count"] == total
assert len(manager.get_state_history("clock")) <= MAX_STATE_HISTORY_PER_PLUGIN
def test_error_transitions_are_capped_too():
"""set_state_with_error() appends to the same list and needs the same cap."""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
for _ in range(MAX_STATE_HISTORY_PER_PLUGIN * 2):
manager.set_state_with_error(
"clock",
PluginState.ENABLED,
{"reason": "update timeout"},
error=RuntimeError("boom"),
)
assert len(manager.get_state_history("clock")) <= MAX_STATE_HISTORY_PER_PLUGIN
def test_history_is_isolated_per_plugin():
"""The cap is per plugin, not shared across the manager."""
manager = PluginStateManager()
for plugin_id in ("clock", "weather"):
manager.set_state(plugin_id, PluginState.ENABLED)
_cycle_updates(manager, plugin_id, 50)
assert len(manager.get_state_history("clock")) == 101
assert len(manager.get_state_history("weather")) == 101
def test_get_state_history_returns_a_copy():
"""Callers must not be able to mutate the manager's internal history."""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
history = manager.get_state_history("clock")
history.clear()
assert len(manager.get_state_history("clock")) == 1
def test_get_state_history_entries_are_copies():
"""Copying the outer list is not enough -- the entries are handed out too.
A caller holding a returned transition must not be able to rewrite the
manager's record of what happened.
"""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
entry = manager.get_state_history("clock")[0]
entry["to"] = "tampered"
entry["error"] = "injected"
stored = manager.get_state_history("clock")[0]
assert stored["to"] == PluginState.ENABLED.value
assert stored["error"] is None
def test_clear_state_drops_history():
"""Unloading a plugin still releases everything it accumulated."""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
_cycle_updates(manager, "clock", 10)
manager.clear_state("clock")
assert manager.get_state_history("clock") == []
assert manager.get_state_info("clock")["state_history_count"] == 0
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
-209
View File
@@ -1,209 +0,0 @@
"""Retention is bounded by age first and by count second.
The cap added in the parent change is a flat entry count, and an entry count
answers the wrong question. What a reader wants from this history is "the last
couple of hours"; how many transitions that is depends entirely on the
plugin's update interval, which on a real board spans 2s to 3600s. A flat 200
entries is 4.2 days of history for the slowest plugin and 3.3 minutes for the
fastest -- so the plugin churning hardest, the one actually worth looking at,
keeps the least.
Trimming by age makes the retained window comparable whatever the cadence, and
the count then serves only as a memory ceiling for pollers fast enough to
produce thousands of transitions inside that window.
"""
import time
import pytest
from src.plugin_system.plugin_state import (
PluginState,
PluginStateManager,
MAX_STATE_HISTORY_PER_PLUGIN,
STATE_HISTORY_MAX_AGE_SECONDS,
)
class FakeClock:
"""A monotonic clock the test drives, so no test has to sleep."""
def __init__(self):
self.t = 1000.0
def __call__(self):
return self.t
def advance(self, seconds):
self.t += seconds
@pytest.fixture
def clock(monkeypatch):
c = FakeClock()
monkeypatch.setattr("src.plugin_system.plugin_state.time.monotonic", c)
return c
def _cycle(manager, plugin_id, clock, interval, cycles):
"""One update cycle: RUNNING on reserve, ENABLED on finish."""
for _ in range(cycles):
manager.set_state(plugin_id, PluginState.RUNNING)
manager.set_state(plugin_id, PluginState.ENABLED)
clock.advance(interval)
def test_transitions_older_than_the_window_are_dropped(clock):
m = PluginStateManager()
_cycle(m, "clock", clock, interval=60, cycles=10)
assert len(m.get_state_history("clock")) == 20
# Nothing happens for longer than the window, then one more cycle.
clock.advance(STATE_HISTORY_MAX_AGE_SECONDS + 1)
_cycle(m, "clock", clock, interval=60, cycles=1)
assert len(m.get_state_history("clock")) == 2, (
"only the transitions inside the window should survive")
def test_every_plugin_keeps_the_same_WINDOW_not_the_same_COUNT(clock):
"""The point of the age policy, stated as the property that distinguishes it.
Run both plugins for three times the retention window. Under a flat count
cap the slow one would still be holding transitions from hours before the
window, because it never produces enough entries to evict them. Under the
age policy each plugin retains its own last two hours and no more --
different entry counts, same span of time.
"""
window = STATE_HISTORY_MAX_AGE_SECONDS
m = PluginStateManager()
_cycle(m, "slow", clock, interval=60, cycles=(3 * window) // 60)
slow = len(m.get_state_history("slow"))
# Assert the property directly rather than a derived count. The guarantee
# is about the SPAN of retained history, not its age against the current
# clock: trimming happens on append, so a plugin that has gone quiet keeps
# its last window until it writes again. That is intentional -- it is
# bounded either way, and a lazy trim costs nothing on the hot path.
stamps = [stamp for stamp, _ in m._state_history["slow"]]
assert stamps[-1] - stamps[0] <= window, (
f"retained history spans {stamps[-1] - stamps[0]:.0f}s, "
f"window is {window}s")
assert slow < 2 * ((3 * window) // 60), (
f"slow plugin kept {slow} entries -- three windows' worth was retained")
clock.t = 1000.0
_cycle(m, "fast", clock, interval=2, cycles=(3 * window) // 2)
fast = len(m.get_state_history("fast"))
# Different counts, and the fast poller keeps more of them -- under a flat
# count cap these would be equal and the fast one would cover minutes.
assert fast > slow, f"fast={fast} slow={slow}"
def test_the_count_ceiling_still_bounds_a_fast_poller(clock):
"""Age alone would let a 2s plugin hold 7,200 entries."""
m = PluginStateManager()
_cycle(m, "flights", clock, interval=2, cycles=STATE_HISTORY_MAX_AGE_SECONDS)
assert len(m.get_state_history("flights")) <= MAX_STATE_HISTORY_PER_PLUGIN
def test_a_burst_inside_the_window_is_capped_not_kept(clock):
"""Transitions with no time between them still cannot grow without bound."""
m = PluginStateManager()
for _ in range(MAX_STATE_HISTORY_PER_PLUGIN * 3):
m.set_state("flapping", PluginState.RUNNING) # clock never advances
assert len(m.get_state_history("flapping")) <= MAX_STATE_HISTORY_PER_PLUGIN
def test_ageing_out_does_not_disturb_the_lifetime_count(clock):
m = PluginStateManager()
_cycle(m, "clock", clock, interval=60, cycles=10)
clock.advance(STATE_HISTORY_MAX_AGE_SECONDS + 1)
_cycle(m, "clock", clock, interval=60, cycles=1)
assert len(m.get_state_history("clock")) == 2
assert m.get_state_info("clock")["state_history_count"] == 22, (
"the lifetime total must survive trimming, it is the flap signal")
def test_the_surviving_entries_are_the_recent_ones(clock):
m = PluginStateManager()
_cycle(m, "clock", clock, interval=60, cycles=5)
clock.advance(STATE_HISTORY_MAX_AGE_SECONDS + 1)
m.set_state("clock", PluginState.ERROR)
history = m.get_state_history("clock")
assert [h["to"] for h in history] == ["error"]
def test_a_monotonic_clock_is_used_not_the_wall_clock(clock):
"""A DST shift or NTP step must not flush the history.
The trim reads time.monotonic(); the human-readable datetime inside each
transition is for display only.
"""
m = PluginStateManager()
_cycle(m, "clock", clock, interval=60, cycles=3)
before = len(m.get_state_history("clock"))
import datetime as real_datetime
class ShiftedDatetime(real_datetime.datetime):
@classmethod
def now(cls, tz=None):
return real_datetime.datetime(1999, 1, 1) # clock jumps backwards
import src.plugin_system.plugin_state as ps
original = ps.datetime
ps.datetime = ShiftedDatetime
try:
m.set_state("clock", PluginState.ENABLED)
finally:
ps.datetime = original
assert len(m.get_state_history("clock")) == before + 1, (
"a wall-clock jump must not trim anything")
def test_get_state_info_is_a_consistent_snapshot():
"""An unload running concurrently must not be observed half-done.
Each field used to be read under its own lock, so clear_state() could
interleave: 'state' read before the removal, 'state_history_count' after,
handing a caller a plugin that is ENABLED with zero transitions. The whole
payload is now built in one critical section.
"""
import threading
m = PluginStateManager()
for _ in range(50):
m.set_state("clock", PluginState.RUNNING)
m.set_state("clock", PluginState.ENABLED)
inconsistent = []
stop = threading.Event()
def reader():
while not stop.is_set():
info = m.get_state_info("clock")
# Either fully present or fully cleared -- never a live state with
# a wiped count.
if info["state"] != PluginState.UNLOADED.value and \
info["state_history_count"] == 0:
inconsistent.append(info)
return
def clearer():
for _ in range(200):
for _ in range(20):
m.set_state("clock", PluginState.ENABLED)
m.clear_state("clock")
t = threading.Thread(target=reader, daemon=True)
t.start()
clearer()
stop.set()
t.join(timeout=5)
assert not inconsistent, f"observed a torn snapshot: {inconsistent[:1]}"
-195
View File
@@ -1,195 +0,0 @@
"""The card helpers the eight scoreboards now share.
These bodies lived in eight byte-identical copies. Moving them here means one
fix reaches every scoreboard — and that a mistake does too, which is what this
file guards. Each case below is one the plugins' own code already handled; the
point is that it keeps handling it.
The functions take ``config``/``logger``/``fonts`` as arguments rather than
reading them off an instance, so a plugin keeps its method and delegates the
body. That is what let all eight adopt this with byte-identical renders.
"""
import logging
import json
import os
import pytest
from src.common import sports_card as C
@pytest.fixture
def log():
return logging.getLogger("test_sports_card")
class TestSettingsLookup:
def test_reads_the_scroll_card_block(self):
cfg = {"scroll_card": {"vs_text": "@"}}
assert C.scroll_card_option(cfg, "vs_text", "VS") == "@"
@pytest.mark.parametrize("cfg", [None, {}, {"scroll_card": None},
{"scroll_card": {"vs_text": None}}])
def test_missing_or_null_falls_back(self, cfg):
"""A null in config means "unset", not "empty string"."""
assert C.scroll_card_option(cfg, "vs_text", "VS") == "VS"
def test_upcoming_center_rejects_unknown_modes(self):
for bad in ("sideways", "", None, 7):
assert C.upcoming_center_mode({"scroll_card": {"upcoming_center": bad}}) == "vs"
assert C.upcoming_center_mode({"scroll_card": {"upcoming_center": "DATE_TIME"}}) == "date_time"
class TestColour:
def test_rgb_list_and_hex_both_work(self):
assert C.element_color({"customization": {"score_text": {"text_color": [1, 2, 3]}}},
"score_text") == (1, 2, 3)
assert C.element_color({"customization": {"score_text": {"text_color": "#ff8000"}}},
"score_text") == (255, 128, 0)
@pytest.mark.parametrize("value", ["nope", "#fff", [1, 2], None, ["a", "b", "c"]])
def test_unusable_colour_falls_back(self, value):
cfg = {"customization": {"score_text": {"text_color": value}}}
assert C.element_color(cfg, "score_text", (9, 9, 9)) == (9, 9, 9)
def test_coerce_rgb_clamps_rather_than_rejecting(self):
assert C.coerce_rgb([300, -5, 20], (0, 0, 0)) == (255, 0, 20)
def test_coerce_rgb_refuses_a_three_character_string(self):
""""123" would otherwise iterate into three digits and yield a colour."""
assert C.coerce_rgb("123", (7, 7, 7)) == (7, 7, 7)
def test_font_colour_is_resolved_by_identity(self):
a, b = object(), object()
cfg = {"customization": {"score_text": {"text_color": [4, 5, 6]}}}
assert C.font_color(cfg, {"score": a, "team": b}, a) == (4, 5, 6)
def test_a_shared_face_gives_up_rather_than_guessing(self):
"""One object used for two elements has no single right colour."""
shared = object()
cfg = {"customization": {"score_text": {"text_color": [4, 5, 6]}}}
assert C.font_color(cfg, {"score": shared, "team": shared}, shared) == (255, 255, 255)
class TestFavourites:
GAME = {"home_abbr": "TB", "away_abbr": "NO", "home_score": "21", "away_score": "17"}
def test_win_loss_and_tie(self):
cfg = {"favorite_teams": ["TB"]}
assert C.favorite_result(cfg, self.GAME) == "win"
assert C.favorite_result({"favorite_teams": ["NO"]}, self.GAME) == "loss"
tied = dict(self.GAME, home_score="3", away_score="3")
assert C.favorite_result(cfg, tied) == "tie"
def test_no_verdict_without_exactly_one_favourite_side(self):
assert C.favorite_result({}, self.GAME) is None
assert C.favorite_result({"favorite_teams": ["TB", "NO"]}, self.GAME) is None
assert C.favorite_result({"favorite_teams": ["SEA"]}, self.GAME) is None
def test_unusable_scores_give_no_verdict(self):
bad = dict(self.GAME, home_score="x")
assert C.favorite_result({"favorite_teams": ["TB"]}, bad) is None
def test_nested_payload_shape_is_read_too(self):
game = {"home_team": {"abbrev": "TB", "score": 9},
"away_team": {"abbrev": "NO", "score": 2}}
assert C.side_score(game, "home") == 9
assert C.side_is_favorite(game, "home", {"TB"}) is True
def test_matches_on_id_where_abbreviations_collide(self):
"""NRL keys favourites by ESPN id; abbreviations are not unique there."""
game = {"home_abbr": "SYD", "home_id": "4321"}
assert C.side_is_favorite(game, "home", {"4321"}) is True
def test_game_and_config_favourites_are_both_used(self):
"""Games carry resolved dynamic groups; config catches later edits."""
game = dict(self.GAME, favorite_teams=["NO"], league="nfl")
assert set(C.favorite_teams_for({"nfl": {"favorite_teams": ["TB"]}}, game)) == {"NO", "TB"}
class TestDateAndTime:
def test_date_formats(self, log):
for fmt, want in [("abbrev", "Sep 5"), ("numeric", "9/5"),
("day_first", "5 Sep"), ("numeric_day_first", "5/9")]:
cfg = {"scroll_card": {"date_format": fmt}}
assert C.format_game_date(cfg, log, "9/5") == want
@pytest.mark.parametrize("raw", ["", "garbage", "13/40", "no/slash/here"])
def test_unparseable_dates_pass_through(self, log, raw):
assert C.format_game_date({}, log, raw) == raw.strip()
def test_24h_conversion(self):
cfg = {"scroll_card": {"time_format": "24h"}}
assert C.format_game_time(cfg, "7:30 PM") == "19:30"
assert C.format_game_time(cfg, "12:00 AM") == "00:00"
assert C.format_game_time(cfg, "12:15 PM") == "12:15"
def test_12h_is_left_alone_and_junk_survives(self):
assert C.format_game_time({}, "7:30 PM") == "7:30 PM"
assert C.format_game_time({"scroll_card": {"time_format": "24h"}}, "soon") == "soon"
def test_a_bad_timezone_falls_back_to_utc(self, log):
"""A typo in config should not blank the card."""
from datetime import timezone
assert C.card_tzinfo({"timezone": "Not/AZone"}, log) is timezone.utc
class TestFontSizing:
def test_snaps_to_the_faces_pixel_grid(self):
assert C.crisp_size("4x6-font.ttf", 6) == 7 # 7px grid
assert C.crisp_size("PressStart2P-Regular.ttf", 10) == 8
assert C.crisp_size("PressStart2P-Regular.ttf", 13) == 16
def test_an_unknown_face_is_never_second_guessed(self):
assert C.crisp_size("SomeUserFont.ttf", 11) == 11
def test_aliases_resolve_before_the_grid_lookup(self):
assert C.crisp_size("four_by_six", 6) == C.crisp_size("4x6-font.ttf", 6)
@pytest.mark.parametrize("desired", [0, -3, None])
def test_unusable_sizes_pass_through_without_raising(self, desired):
"""None reached this in the field; football's variant raised TypeError."""
assert C.crisp_size("4x6-font.ttf", desired) == desired
def test_schema_cache_is_keyed_per_schema_not_globally(self, tmp_path):
"""Two plugins declaring different defaults must not share an answer."""
a, b = tmp_path / "a.json", tmp_path / "b.json"
for path, size in ((a, 11), (b, 22)):
path.write_text(json.dumps({"properties": {"customization": {"properties": {
"score_text": {"properties": {"font_size": {"default": size}}}}}}}))
assert C.schema_font_size(str(a), "score_text") == 11
assert C.schema_font_size(str(b), "score_text") == 22
def test_a_missing_schema_is_not_an_error(self, tmp_path):
assert C.schema_font_size(str(tmp_path / "nope.json"), "score_text") is None
def test_a_configured_size_matching_the_schema_default_is_not_a_choice(self, tmp_path):
"""The web UI writes the whole default block on every save, so
font_size == schema default carries no intent and must not pin the
install to an off-grid size forever."""
schema = tmp_path / "s.json"
schema.write_text(json.dumps({"properties": {"customization": {"properties": {
"score_text": {"properties": {"font_size": {"default": 10}}}}}}}))
got = C.resolve_font_size(str(schema), {"font_size": 10}, "score_text", 10,
"PressStart2P-Regular.ttf")
assert got == 8, "a default-valued size should snap to the grid"
def test_a_real_choice_wins(self, tmp_path):
schema = tmp_path / "s.json"
schema.write_text(json.dumps({"properties": {"customization": {"properties": {
"score_text": {"properties": {"font_size": {"default": 10}}}}}}}))
got = C.resolve_font_size(str(schema), {"font_size": 13}, "score_text", 10,
"PressStart2P-Regular.ttf")
assert got == 13, "an explicit size the user chose is not second-guessed"
class TestTables:
def test_every_font_key_maps_to_an_element(self):
assert set(C.ELEMENT_FOR_FONT) == {"score", "time", "team", "status", "detail", "rank"}
def test_result_colours_cover_every_verdict(self):
assert set(C.FAVORITE_RESULT_COLOR_DEFAULTS) == {"win", "loss", "tie"}
def test_month_and_weekday_tables_are_complete(self):
assert len(C.MONTH_ABBR) == 12 and len(C.WEEKDAY_ABBR) == 7
-258
View File
@@ -1,258 +0,0 @@
"""The shared card geometry, exercised against a host that supplies only what
the mixin's contract names.
The point of these is the contract, not the arithmetic. The mixin reaches for
``display_width``, ``fonts``, ``config`` and six ``sports_card`` delegations
through ``self``, and the eight plugins are what actually provide them. A
stub host that provides exactly the documented surface and nothing else is
what catches the mixin quietly growing a dependency the plugins do not have.
"""
import pytest
from PIL import Image, ImageDraw, ImageFont
from src.common.sports_game_renderer import SportsGameRendererMixin
class Host(SportsGameRendererMixin):
"""The documented contract, and not one attribute more."""
def __init__(self, width=128, height=32, config=None, scroll=None):
self.display_width = width
self.display_height = height
self.config = config or {}
self.logger = _Logger()
self._team_rankings_cache = {}
self._scroll = scroll or {}
font = ImageFont.load_default()
self.fonts = {'score': font, 'time': font, 'detail': font}
self.drawn = []
# -- the six sports_card delegations the mixin calls --
def _scroll_card_option(self, key, default=None):
return self._scroll.get(key, default)
def _upcoming_center_mode(self):
return self._scroll.get('upcoming_center', 'vs')
def _vs_text(self):
return self._scroll.get('vs_text', 'VS')
def _element_color(self, element):
return (255, 255, 255)
def _format_game_date(self, raw, game):
return raw
def _format_game_time(self, raw):
return raw
# -- the one hook whose body genuinely varies per plugin --
def _draw_text_with_outline(self, draw, text, position, font,
fill=None, outline_color=(0, 0, 0)):
self.drawn.append((text, position))
class _Logger:
def debug(self, *a, **k):
pass
def _draw():
return ImageDraw.Draw(Image.new("RGB", (256, 64)))
class TestCenterGap:
def test_explicit_center_gap_wins_outright(self):
assert Host(scroll={'center_gap': 31})._center_gap_width() == 31
def test_explicit_zero_restores_edge_to_edge_logos(self):
# 0 is a real setting, not a falsy miss -- the guard is `>= 0`.
assert Host(scroll={'center_gap': 0})._center_gap_width() == 0
def test_otherwise_it_scales_with_card_width_within_the_clamp(self):
h = Host(width=512)
# 512 * 0.28 = 143, clamped to the 40px ceiling.
assert h._center_gap_width() >= h.CENTER_GAP_MIN_PX
def test_the_gap_never_ends_up_narrower_than_the_score(self):
# This is the bug the measurement exists to prevent: a derived gap
# smaller than the rendered score drew the score over the logos.
h = Host(width=64)
assert h._center_gap_width() >= h._score_reserve_width()
def test_a_junk_ratio_falls_back_to_the_floor(self):
h = Host(scroll={'center_gap_ratio': 'wide'})
assert h._center_gap_width() == h.CENTER_GAP_MIN_PX
class TestNonFiniteSettings:
"""inf reaches int() and raises OverflowError, which the old
`except (TypeError, ValueError)` did not catch -- so one bad config value
aborted the entire card render rather than falling back."""
@pytest.mark.parametrize("bad", [float("inf"), float("-inf")])
def test_a_non_finite_center_gap_falls_back(self, bad):
h = Host(scroll={'center_gap': bad})
assert h._center_gap_width() >= h.CENTER_GAP_MIN_PX
@pytest.mark.parametrize("bad", [float("inf"), float("-inf"), float("nan")])
def test_a_non_finite_ratio_falls_back_to_the_floor(self, bad):
h = Host(scroll={'center_gap_ratio': bad})
assert h._center_gap_width() == h.CENTER_GAP_MIN_PX
@pytest.mark.parametrize("bad", [float("inf"), float("-inf")])
def test_non_finite_clamp_bounds_fall_back(self, bad):
h = Host(scroll={'center_gap_min': bad, 'center_gap_max': bad})
assert h._center_gap_width() == h.CENTER_GAP_MIN_PX
@pytest.mark.parametrize("bad", [float("inf"), float("-inf"), "inf", "-inf", "nan"])
def test_a_non_finite_layout_offset_gives_the_default(self, bad):
cfg = {'customization': {'layout': {'score': {'x_offset': bad}}}}
assert Host(config=cfg)._layout_offset('score', 'x_offset', 7) == 7
def test_a_finite_value_is_still_honoured(self):
# The guard must not swallow ordinary settings.
assert Host(scroll={'center_gap': 31})._center_gap_width() == 31
cfg = {'customization': {'layout': {'score': {'x_offset': -3}}}}
assert Host(config=cfg)._layout_offset('score', 'x_offset', 7) == -3
class TestScoreReserve:
def test_it_measures_the_probe_plus_both_gutters(self):
h = Host()
assert h._score_reserve_width() > 2 * h._SCORE_LOGO_GUTTER_PX
def test_a_wider_probe_reserves_more(self):
class Wide(Host):
_SCORE_PROBE = "000-000"
assert Wide()._score_reserve_width() > Host()._score_reserve_width()
def test_an_unmeasurable_font_reserves_nothing_rather_than_raising(self):
h = Host()
h.fonts = {'score': object()}
assert h._score_reserve_width() == 0
class TestLogoSlot:
def test_the_slot_is_what_is_left_after_the_gap(self):
h = Host(width=128, scroll={'center_gap': 40})
assert h._logo_slot_width() == 44
def test_it_is_not_capped_at_the_card_height(self):
# The height cap is what froze logos at 46px on a 128px card.
h = Host(width=512, height=32, scroll={'center_gap': 40})
assert h._logo_slot_width() > h.display_height
def test_a_gap_wider_than_the_card_still_leaves_a_usable_slot(self):
assert Host(width=64, scroll={'center_gap': 200})._logo_slot_width() == 8
def test_the_cache_key_is_scoped_to_the_slot_not_just_the_name(self):
# One cache dict is shared by renderers of different card widths.
narrow = Host(width=64, scroll={'center_gap': 20})._logo_cache_key("NYY")
wide = Host(width=256, scroll={'center_gap': 20})._logo_cache_key("NYY")
assert narrow != wide
class TestLayoutOffset:
def _host(self, value):
return Host(config={'customization': {'layout': {'score': {'x_offset': value}}}})
def test_it_reads_the_same_block_as_the_full_screen_scorebug(self):
assert self._host(5)._layout_offset('score', 'x_offset') == 5
def test_a_string_offset_from_the_web_ui_is_coerced(self):
assert self._host("-3")._layout_offset('score', 'x_offset') == -3
def test_a_bool_is_not_silently_an_offset_of_one(self):
assert self._host(True)._layout_offset('score', 'x_offset', 9) == 9
@pytest.mark.parametrize("cfg", [{}, {'customization': {}},
{'customization': {'layout': {}}}])
def test_a_missing_block_gives_the_default(self, cfg):
assert Host(config=cfg)._layout_offset('score', 'x_offset', 7) == 7
def test_an_unparseable_offset_gives_the_default(self):
assert self._host("left")._layout_offset('score', 'x_offset', 4) == 4
class TestUpcomingCenter:
def test_none_draws_nothing(self):
h = Host(scroll={'upcoming_center': 'none'})
h._draw_upcoming_center(_draw(), {})
assert h.drawn == []
def test_vs_is_the_default_and_never_a_score(self):
# An upcoming game has not started; the extractor's 0-0 is noise.
h = Host()
h._draw_upcoming_center(_draw(), {'home_score': 0, 'away_score': 0})
assert [t for t, _ in h.drawn] == ['VS']
def test_an_empty_vs_string_draws_nothing(self):
h = Host(scroll={'vs_text': ''})
h._draw_upcoming_center(_draw(), {})
assert h.drawn == []
def test_date_time_stacks_both_lines(self):
h = Host(scroll={'upcoming_center': 'date_time'})
h._draw_upcoming_center(_draw(), {'game_date': 'Sep 19', 'game_time': '7:00 PM'})
assert [t for t, _ in h.drawn] == ['Sep 19', '7:00 PM']
def test_hiding_both_lines_draws_nothing(self):
h = Host(scroll={'upcoming_center': 'date_time',
'show_date': False, 'show_time': False})
h._draw_upcoming_center(_draw(), {'game_date': 'Sep 19', 'game_time': '7:00 PM'})
assert h.drawn == []
class TestUpcomingStatus:
def test_time_on_top_and_date_below_by_default(self):
h = Host()
h._draw_upcoming_game_status(_draw(), {'game_date': 'Sep 19', 'game_time': '7:00 PM'})
assert [t for t, _ in h.drawn] == ['7:00 PM', 'Sep 19']
def test_swap_date_time_reverses_them(self):
h = Host(scroll={'swap_date_time': True})
h._draw_upcoming_game_status(_draw(), {'game_date': 'Sep 19', 'game_time': '7:00 PM'})
assert [t for t, _ in h.drawn] == ['Sep 19', '7:00 PM']
def test_it_stays_out_of_the_way_when_the_centre_already_has_them(self):
# Otherwise the date and time print twice on the same card.
h = Host(scroll={'upcoming_center': 'date_time'})
h._draw_upcoming_game_status(_draw(), {'game_date': 'Sep 19', 'game_time': '7:00 PM'})
assert h.drawn == []
def test_the_bottom_line_is_measured_not_a_fixed_offset(self):
# A fixed -7 ran "Sep 19" past the card wherever the detail font is
# 10px rather than 6px.
h = Host(height=64)
h._draw_upcoming_game_status(_draw(), {'game_date': 'Sep 19', 'game_time': '7:00 PM'})
bottom_y = h.drawn[1][1][1]
assert 0 <= bottom_y < h.display_height
class TestRankings:
def test_set_rankings_cache_replaces_the_cache(self):
h = Host()
h.set_rankings_cache({'UGA': 1})
assert h._team_rankings_cache == {'UGA': 1}
class TestContract:
def test_the_mixin_carries_no_state_of_its_own(self):
# Adoption must be one line on the class statement; a mixin with an
# __init__ would force eight constructors to cooperate.
assert '__init__' not in SportsGameRendererMixin.__dict__
def test_a_host_providing_the_documented_surface_needs_nothing_more(self):
# Host defines exactly what the module docstring names. If the mixin
# grows a new self.* dependency, this is what fails.
h = Host()
h._center_gap_width()
h._logo_slot_width()
h._logo_cache_key("X")
h._layout_offset('score', 'x_offset')
h._upcoming_date_and_time({})
h._draw_upcoming_center(_draw(), {})
h._draw_upcoming_game_status(_draw(), {})
h.set_rankings_cache({})
-321
View File
@@ -1,321 +0,0 @@
"""The shared sports.py mixins: their host contract, and _plugin_dir.
Two things are worth testing here and the rest is not. The 45 method bodies
moved verbatim from the plugins, so they are covered by the plugins' own tests
and by 176 byte-identical safety-harness renders. What is genuinely new is:
1. The contract. Every ``self.<CONSTANT>`` a mixin reads must be defined on the
mixin, or a host that does not happen to declare it raises AttributeError at
runtime. Two were missed on the first pass (_QUALITY_CHOICES and
_RANKING_COVERAGE_SECONDS); the eight plugins all declare them, so nothing
failed -- it would only have bitten a ninth. The test derives the list rather
than restating it, so the next omission fails here instead of in the field.
2. ``_plugin_dir``. This is the only line of genuinely new logic in the move. In
sports.py these methods found config_schema.json with ``__file__``; here that
is src/common/, so the plugin directory has to be recovered from the
instance -- and getting it wrong is silent, costing grid-snapped font sizes
(measured at 81% anti-aliased edges) rather than raising.
"""
import ast
import os
import sys
import types
from abc import ABC
import pytest
from src.common import sports_shared
from src.common.sports_shared import (
SportsCoreSharedMixin, SportsLiveSharedMixin, SportsRecentSharedMixin)
MIXINS = (SportsCoreSharedMixin, SportsLiveSharedMixin, SportsRecentSharedMixin)
def _constants_read_by_mixins():
"""Every ALL-CAPS ``self.X`` the mixin bodies read, found by parsing them."""
tree = ast.parse(open(sports_shared.__file__).read())
names = set()
for node in ast.walk(tree):
if (isinstance(node, ast.Attribute)
and isinstance(node.value, ast.Name)
and node.value.id == "self"
and node.attr.upper() == node.attr):
names.add(node.attr)
return names
class TestHostContract:
def test_every_constant_read_is_also_defined(self):
# Otherwise a host that does not declare it raises AttributeError the
# first time the code path runs -- which for these is mid-render.
missing = sorted(
name for name in _constants_read_by_mixins()
if not any(hasattr(m, name) for m in MIXINS))
assert missing == [], (
f"read but never defined on a mixin: {missing}. Give each a default "
f"on SportsCoreSharedMixin and document it in the module docstring.")
@pytest.mark.parametrize("name,expected", [
("_QUALITY_CHOICES", frozenset({"any", "ranked"})),
("_RANKING_COVERAGE_SECONDS", 3600),
("_SCORE_PROBE_TEXT", "00-00"),
("_FONT_DESIGN_HEIGHT", 32),
])
def test_defaults_match_what_the_plugins_ship(self, name, expected):
# The eight plugins declare their own copies, which shadow these. The
# values must still agree, or a ninth plugin inheriting the default
# behaves differently from the eight.
assert getattr(SportsCoreSharedMixin, name) == expected
def test_only_the_recent_mixin_carries_a_constructor(self):
# SportsCore and SportsLive keep their own __init__ -- those differ per
# plugin. SportsRecent.__init__ was one of the 48 byte-identical bodies,
# so it moved with the rest; that is deliberate, not an oversight.
assert "__init__" not in SportsCoreSharedMixin.__dict__
assert "__init__" not in SportsLiveSharedMixin.__dict__
assert "__init__" in SportsRecentSharedMixin.__dict__
def test_the_recent_constructor_still_chains_to_the_host(self):
"""Its zero-arg super() binds to where it is DEFINED, not where it is used.
Moving a body containing bare ``super()`` is the one move that can
change meaning: the compiler closes over __class__ = the defining class,
so after the move that is SportsRecentSharedMixin rather than the
plugin's SportsRecent. It still works only because the mixin is listed
first, leaving the host class next in the MRO -- adopt it in the other
order and the chain silently skips the host's __init__.
"""
calls = []
class Host:
def __init__(self, config, display_manager, cache_manager, logger, sport_key):
calls.append(sport_key)
self.mode_config = {}
class Recent(SportsRecentSharedMixin, Host):
pass
inst = Recent({}, None, None, None, "nhl")
assert calls == ["nhl"], "the host constructor must still run"
assert inst.current_game_index == 0
assert inst.update_interval == 3600
assert inst._zero_clock_timestamps == {}
def test_adopting_the_recent_mixin_second_would_skip_the_host(self):
# The failure mode the ordering above prevents, pinned so nobody
# "tidies" the base list.
calls = []
class Host:
def __init__(self, *a):
calls.append(a)
self.mode_config = {}
class Wrong(Host, SportsRecentSharedMixin):
pass
Wrong({}, None, None, None, "nhl")
# Host.__init__ wins and the mixin's setup never runs at all.
assert not hasattr(Wrong({}, None, None, None, "nhl"), "current_game_index")
class _Host(SportsCoreSharedMixin):
pass
def _write_plugin(tmp_path, name="fakeplug", schema=True):
"""A throwaway package on sys.path, with or without a config_schema.json."""
d = tmp_path / name
d.mkdir()
(d / "__init__.py").write_text("")
(d / "mod.py").write_text("class Leaf:\n pass\n")
if schema:
(d / "config_schema.json").write_text(
'{"properties": {"customization": {"properties": '
'{"score": {"properties": {"font_size": {"default": 16}}}}}}}')
return d
class TestPluginDir:
def test_it_finds_the_directory_holding_config_schema_json(self, tmp_path, monkeypatch):
d = _write_plugin(tmp_path)
monkeypatch.syspath_prepend(str(tmp_path))
mod = __import__("fakeplug.mod", fromlist=["Leaf"])
host = type("H", (mod.Leaf, SportsCoreSharedMixin), {})()
assert host._plugin_dir() == str(d)
def test_a_class_built_by_type_still_resolves(self, tmp_path, monkeypatch):
# SportsCore is an ABC, so type(name, bases, ns) reports __module__ as
# "abc" rather than the plugin -- which is exactly what the plugins'
# own tests build. Walking the MRO is what steps past it.
d = _write_plugin(tmp_path, "abcplug")
monkeypatch.syspath_prepend(str(tmp_path))
mod = __import__("abcplug.mod", fromlist=["Leaf"])
class Base(SportsCoreSharedMixin, mod.Leaf, ABC):
pass
synthetic = type("Probe", (Base,), {})
assert synthetic.__module__ == "abc", "precondition: the trap this guards"
assert synthetic.__new__(synthetic)._plugin_dir() == str(d)
def test_it_returns_none_when_no_schema_is_anywhere_on_the_mro(self, tmp_path, monkeypatch):
d = _write_plugin(tmp_path, "noschema", schema=False)
monkeypatch.syspath_prepend(str(tmp_path))
mod = __import__("noschema.mod", fromlist=["Leaf"])
host = type("H", (mod.Leaf, SportsCoreSharedMixin), {})()
# None rather than a wrong guess: _schema_font_size then caches empty
# and every element keeps its own default.
assert host._plugin_dir() is None
def test_it_never_returns_the_core_module_directory(self):
# The bug this replaced: __file__ pointed at src/common/, so the schema
# was never found and font sizes silently stopped snapping to the grid.
host = _Host()
core_common = os.path.dirname(os.path.abspath(sports_shared.__file__))
assert host._plugin_dir() != core_common
def test_a_module_with_no_file_is_skipped_not_crashed_on(self, monkeypatch):
# Namespace packages and some frozen/dynamic modules have no __file__.
ghost = types.ModuleType("ghost_no_file")
if hasattr(ghost, "__file__"):
del ghost.__file__
monkeypatch.setitem(sys.modules, "ghost_no_file", ghost)
cls = type("H", (SportsCoreSharedMixin,), {"__module__": "ghost_no_file"})
assert cls.__new__(cls)._plugin_dir() is None
class TestSchemaFontSize:
def test_it_reads_the_plugin_schema_not_the_cores(self, tmp_path, monkeypatch):
d = _write_plugin(tmp_path, "sizeplug")
monkeypatch.syspath_prepend(str(tmp_path))
mod = __import__("sizeplug.mod", fromlist=["Leaf"])
host = type("H", (mod.Leaf, SportsCoreSharedMixin), {})()
assert host._schema_font_size("score") == 16
def test_an_unknown_element_is_none(self, tmp_path, monkeypatch):
_write_plugin(tmp_path, "unkplug")
monkeypatch.syspath_prepend(str(tmp_path))
mod = __import__("unkplug.mod", fromlist=["Leaf"])
host = type("H", (mod.Leaf, SportsCoreSharedMixin), {})()
assert host._schema_font_size("nonesuch") is None
def test_an_empty_key_is_none_without_touching_the_disk(self):
assert _Host()._schema_font_size("") is None
def test_a_missing_schema_degrades_to_none_rather_than_raising(self, tmp_path, monkeypatch):
_write_plugin(tmp_path, "bareplug", schema=False)
monkeypatch.syspath_prepend(str(tmp_path))
mod = __import__("bareplug.mod", fromlist=["Leaf"])
host = type("H", (mod.Leaf, SportsCoreSharedMixin), {})()
assert host._schema_font_size("score") is None
class _LiveHost(SportsLiveSharedMixin):
"""The documented contract for the live mixin, and nothing else."""
def __init__(self, no_data_interval=300, stale_game_timeout=600, over=()):
self.no_data_interval = no_data_interval
self.stale_game_timeout = stale_game_timeout
self.game_update_timestamps = {}
self._over = set(over)
class _L:
def __getattr__(self, _n):
return lambda *a, **k: None
self.logger = _L()
def _is_game_really_over(self, game):
return game.get("id") in self._over
class TestLiveMixin:
"""These three moved to the core, so they are tested here.
They were already covered by hockey's and lacrosse's own tests, but those
two plugins disable live mode in their safety-harness fixtures, so the 176
renders never exercise this path. Testing the mixin directly means the
coverage no longer depends on which plugin happens to have a unit test.
"""
def test_a_stale_game_is_dropped_and_forgotten(self):
h = _LiveHost(stale_game_timeout=600)
import time as _t
h.game_update_timestamps["g1"] = {"last_seen": _t.time() - 5000}
games = [{"id": "g1", "home_abbr": "H", "away_abbr": "A"}]
h._detect_stale_games(games)
assert games == []
assert "g1" not in h.game_update_timestamps, "its timestamp must go too"
def test_a_fresh_game_survives(self):
h = _LiveHost(stale_game_timeout=600)
import time as _t
h.game_update_timestamps["g1"] = {"last_seen": _t.time() - 5}
games = [{"id": "g1"}]
h._detect_stale_games(games)
assert len(games) == 1
def test_a_game_never_seen_is_not_treated_as_stale(self):
# last_seen 0 means "no reading", not "seen at the epoch".
h = _LiveHost()
games = [{"id": "g1"}]
h._detect_stale_games(games)
assert len(games) == 1
def test_a_game_with_no_id_is_left_alone(self):
h = _LiveHost()
games = [{"home_abbr": "H"}]
h._detect_stale_games(games)
assert len(games) == 1
def test_a_finished_game_is_dropped_even_when_fresh(self):
h = _LiveHost(over=("g2",))
games = [{"id": "g1"}, {"id": "g2"}]
h._detect_stale_games(games)
assert [g["id"] for g in games] == ["g1"]
def test_removing_several_does_not_skip_any(self):
# It iterates a copy for exactly this reason; mutating the live list
# while looping would step over the element after each removal.
h = _LiveHost(over=("g1", "g2", "g3"))
games = [{"id": "g1"}, {"id": "g2"}, {"id": "g3"}]
h._detect_stale_games(games)
assert games == []
def test_the_idle_interval_escalates_with_the_empty_streak(self):
h = _LiveHost(no_data_interval=60)
h.live_idle_max_interval = 100000
base = h._idle_live_interval()
h._empty_live_streak = 6
short = h._idle_live_interval()
h._empty_live_streak = 24
long = h._idle_live_interval()
assert base < short < long
def test_the_ceiling_bounds_even_the_unescalated_interval(self):
# base > ceiling is a reachable config: the two settings are
# independent integers with no cross-validation. Returning base
# unclamped made the wait SHRINK as the streak grew.
h = _LiveHost(no_data_interval=3600)
h.live_idle_max_interval = 900
h._empty_live_streak = 0
assert h._idle_live_interval() == 900
h._empty_live_streak = 24
assert h._idle_live_interval() == 900
def test_finding_a_live_game_resets_the_streak(self):
h = _LiveHost()
h._note_live_fetch(False)
h._note_live_fetch(False)
assert h._empty_live_streak == 2
h._note_live_fetch(True)
assert h._empty_live_streak == 0
def test_the_streak_starts_from_absent_state(self):
# The host is not required to pre-declare _empty_live_streak.
h = _LiveHost()
assert not hasattr(h, "_empty_live_streak")
h._note_live_fetch(False)
assert h._empty_live_streak == 1
-310
View File
@@ -1,310 +0,0 @@
"""What happens to an adopted sports plugin on each core it can meet.
B5 moved the eight scoreboards onto `src.common.sports_scroll` behind a guarded
import, keeping a bundled copy as the fallback. B6 deletes those copies. The
two phases make *different* promises, and only the second one is obvious:
| | bundled copy present | bundled copy removed |
|--------------------|---------------------------------|-----------------------------|
| **pinned old core**| loads -- B5's whole guarantee | ERROR, naming the module |
| **current core** | loads, using core code | loads, using core code |
The top-left cell is the one worth having: nothing else in this suite proves an
adopted plugin still runs on a core that predates the module, and that claim is
the entire basis for having shipped B5 ahead of B6's gate.
The bottom-left cell is the B6 failure mode, and it is asserted through
`PluginManager.load_plugin` rather than a bare import on purpose. The manager
catches the `ModuleNotFoundError`, so nothing propagates to a caller: a test
that expected `pytest.raises` would pass against a core where the module is
merely *broken* rather than absent, and would say nothing about what the user
actually experiences. What they get is a plugin parked in ERROR and one log
line -- which is precisely why B6 needs the install gate rather than trusting
the failure to be noticed.
This covers the load path. The install/update gate -- which is what should stop
a sunset plugin reaching an old core in the first place -- is the other half of
the guarantee and is tested in test_plugin_compatibility_gate.py.
See docs/SPORTS_UNIFICATION.md, "B6 -- why the sunset needs more than a version
floor".
"""
import itertools
import json
import sys
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
project_root = Path(__file__).parent.parent
if str(project_root) not in sys.path:
sys.path.insert(0, str(project_root))
from src.plugin_system.plugin_manager import PluginManager
from src.plugin_system.plugin_state import PluginState
_PLUGIN_IDS = itertools.count()
CORE_MODULE = "src.common.sports_scroll"
# Only the leaf. A pre-3.2.0 core still ships `src/common/` -- scroll_helper
# and friends live there -- and it is `sports_scroll.py` alone that is absent.
# Hiding the whole package would be a different, harsher core than any that
# shipped, and it would make the load failure name the package rather than the
# module, which is the thing a reader needs to see.
HIDDEN = (CORE_MODULE,)
class _PinnedOldCore:
"""Make `src.common.sports_scroll` un-importable for the duration.
A meta_path finder rather than a monkeypatched `__import__`: the plugin is
executed by the real loader through `exec_module`, so the block has to live
in the import system itself to be reached.
"""
def __init__(self, *names):
self.names = set(names)
self._saved = {}
def find_spec(self, fullname, path=None, target=None):
if fullname in self.names:
raise ModuleNotFoundError(f"No module named {fullname!r}", name=fullname)
return None
def __enter__(self):
for name in list(sys.modules):
if name in self.names:
self._saved[name] = sys.modules.pop(name)
sys.meta_path.insert(0, self)
return self
def __exit__(self, *exc):
sys.meta_path.remove(self)
sys.modules.update(self._saved)
return False
# The two source shapes, kept as literals rather than copied from a plugin at
# runtime: this file lives in the core repo and must not depend on a plugin
# checkout being present, and pinning the shapes here means a plugin that
# drifts away from one is a visible edit, not a silently weakened test.
# B5, as the eight scoreboards ship today: prefer the core module, fall back to
# the bundled copy. The except clause is narrow on purpose -- a bare
# `except ImportError` would also swallow a failure raised *inside* a core
# module that is present, quietly loading the legacy copy and hiding a broken
# core install.
ADOPTED_WITH_FALLBACK = '''
_USING_CORE_SCROLL = False
try:
from src.common.sports_scroll import SportsScrollDisplay as _Base
_USING_CORE_SCROLL = True
except ModuleNotFoundError as exc:
if exc.name not in {"src", "src.common", "src.common.sports_scroll"}:
raise
_Base = None
if not _USING_CORE_SCROLL:
from scroll_display_legacy import LegacyScrollDisplay as _Base
'''
# B6, once the copies are deleted: there is nothing to fall back to, so the
# guard goes with them. Keeping the try/except while removing the file it
# falls back to would only mislabel the failure -- the plugin would report a
# missing `scroll_display_legacy` and say nothing about the core module that
# is actually absent.
SUNSET_NO_FALLBACK = '''
from src.common.sports_scroll import SportsScrollDisplay as _Base
_USING_CORE_SCROLL = True
'''
PLUGIN_BODY = '''
from src.plugin_system.base_plugin import BasePlugin
class SunsetProbe(BasePlugin):
"""Records which scroll implementation the guarded import selected."""
using_core_scroll = _USING_CORE_SCROLL
scroll_base = _Base
def update(self):
pass
def display(self, force_clear=False):
pass
'''
LEGACY_COPY = '''
class LegacyScrollDisplay:
"""Stands in for the bundled pre-3.2.0 implementation."""
'''
def _write_plugin(plugins_dir: Path, plugin_id: str, *, bundled_copy: bool) -> Path:
path = plugins_dir / plugin_id
path.mkdir(parents=True)
(path / "manifest.json").write_text(
json.dumps({
"id": plugin_id,
"name": "Sunset Probe",
"version": "1.0.0",
"entry_point": "manager.py",
"class_name": "SunsetProbe",
"display_modes": ["sunset_probe"],
}),
encoding="utf-8",
)
shape = ADOPTED_WITH_FALLBACK if bundled_copy else SUNSET_NO_FALLBACK
(path / "manager.py").write_text(shape + PLUGIN_BODY, encoding="utf-8")
if bundled_copy:
(path / "scroll_display_legacy.py").write_text(LEGACY_COPY, encoding="utf-8")
return path
@pytest.fixture
def load(tmp_path):
"""Load a synthetic adopted plugin through the real PluginManager.
Returns a callable taking the two axes of the matrix and handing back the
manager, so the caller can ask it for state and recorded error.
"""
plugins_dir = tmp_path / "plugin-repos"
plugins_dir.mkdir()
def _load(*, core_has_module: bool, bundled_copy: bool, hide=HIDDEN):
# A distinct id per cell, from a counter that spans the whole session.
# The loader names plugin modules after the plugin id and sys.modules
# is process-global, so a per-test counter would hand the second test
# the first test's already-imported module -- which passes or fails on
# the wrong plugin's import.
plugin_id = f"sunset-probe-{next(_PLUGIN_IDS)}"
_write_plugin(plugins_dir, plugin_id, bundled_copy=bundled_copy)
with patch('src.common.permission_utils.ensure_directory_permissions'):
manager = PluginManager(
plugins_dir=str(plugins_dir),
config_manager=MagicMock(),
display_manager=MagicMock(),
cache_manager=MagicMock(),
font_manager=MagicMock(),
)
manager.discover_plugins()
if core_has_module:
ok = manager.load_plugin(plugin_id)
else:
with _PinnedOldCore(*hide):
ok = manager.load_plugin(plugin_id)
return manager, plugin_id, ok
return _load
def _assert_loaded(manager, plugin_id, ok):
assert ok is True, (
f"load_plugin returned False; state is "
f"{manager.state_manager.get_state(plugin_id)}, error "
f"{manager.state_manager.get_error_info(plugin_id)}"
)
assert manager.state_manager.get_state(plugin_id) is not PluginState.ERROR
class TestBundledCopyPresent:
"""B5's shape: the guarded import with the fallback still shipped."""
def test_old_core_falls_back_and_still_loads(self, load):
"""The claim that made it safe to ship B5 before B6's gate."""
manager, plugin_id, ok = load(core_has_module=False, bundled_copy=True)
_assert_loaded(manager, plugin_id, ok)
plugin = manager.plugins[plugin_id]
assert plugin.using_core_scroll is False, (
"the core module was hidden, so the plugin must be on its bundled copy"
)
assert plugin.scroll_base.__name__ == "LegacyScrollDisplay"
def test_current_core_prefers_the_core_module(self, load):
manager, plugin_id, ok = load(core_has_module=True, bundled_copy=True)
_assert_loaded(manager, plugin_id, ok)
plugin = manager.plugins[plugin_id]
assert plugin.using_core_scroll is True, (
"the bundled copy must not win while the core module is importable"
)
assert plugin.scroll_base.__name__ == "SportsScrollDisplay"
def test_a_core_without_the_package_at_all_still_falls_back(self, load):
"""The guard's other accepted names.
Its except clause accepts `src` and `src.common` as well as the module
itself, so those branches exist in all eight shipped plugins. No core
that old is likely still running, but the code claiming to handle it is
real and nothing else exercises it -- an untested branch in a fallback
is exactly the kind that rots unnoticed until the fallback is needed.
"""
manager, plugin_id, ok = load(
core_has_module=False, bundled_copy=True, hide=(CORE_MODULE, "src.common")
)
_assert_loaded(manager, plugin_id, ok)
assert manager.plugins[plugin_id].using_core_scroll is False
class TestBundledCopyRemoved:
"""B6's shape: the copies are gone and only the core module remains."""
def test_current_core_still_loads(self, load):
manager, plugin_id, ok = load(core_has_module=True, bundled_copy=False)
_assert_loaded(manager, plugin_id, ok)
assert manager.plugins[plugin_id].using_core_scroll is True
def test_old_core_errors_and_records_the_missing_module(self, load):
"""The B6 failure mode, as the user meets it.
Not `pytest.raises`: load_plugin catches it, so nothing reaches a
caller. The observable consequences are the ERROR state and the
recorded error -- and the error has to name the module, or whoever
reads the log cannot tell a missing core module from any other
import failure.
"""
manager, plugin_id, ok = load(core_has_module=False, bundled_copy=False)
assert ok is False, "a plugin with no scroll implementation must not load"
assert manager.state_manager.get_state(plugin_id) is PluginState.ERROR
assert plugin_id not in manager.plugins, (
"a plugin that failed to load must not be left registered"
)
info = manager.state_manager.get_error_info(plugin_id)
assert info is not None, "ERROR state recorded no error to explain it"
assert info['error_type'] == 'ModuleNotFoundError', info
assert CORE_MODULE in info['error'], (
f"the recorded error must name the module that was missing, got {info['error']!r}"
)
def test_the_matrix_has_one_failing_cell(load):
"""Guards the shape of the table itself.
Each cell above is asserted on its own, so a change that broke two of them
in compensating ways could leave every individual test passing. This says
the outcome depends on both axes and fails in exactly one combination.
"""
outcomes = {
(core, bundled): load(core_has_module=core, bundled_copy=bundled)[2]
for core in (True, False)
for bundled in (True, False)
}
assert outcomes == {
(True, True): True,
(True, False): True,
(False, True): True,
(False, False): False,
}, outcomes
@@ -1,92 +0,0 @@
"""/api/v3/system/status must report MemAvailable, not just used/total.
"Memory used %" cannot tell a healthy board from one about to fail. Page cache
counts as used and is reclaimable on demand, so a Pi can read 70% used and be
perfectly fine, or read the same and be minutes from trouble. MemAvailable is
the kernel's own estimate of what a new allocation can actually obtain, and it
is the number that tracked the failure on a 1GB Pi 3B+: healthy running sat at
500MB+, the crash happened at 73MB, and by then fork() was failing -- sshd
could not spawn a session and systemd could not respawn the display, while the
kernel carried on answering pings.
psutil.virtual_memory().available is MemAvailable on Linux. total - used is not
a substitute: they diverge exactly when unreclaimable memory (shmem, tmpfs) is
in play, which is when the distinction matters.
"""
import json
import sys
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
sys.path.insert(0, str(Path(__file__).parent.parent))
MB = 1024 * 1024
@pytest.fixture
def client():
pytest.importorskip("psutil")
app = Flask(__name__)
app.config["TESTING"] = True
from web_interface.blueprints.api_v3 import api_v3
for attr in ("config_manager", "plugin_manager", "cache_manager"):
setattr(api_v3, attr, MagicMock())
if "api_v3" not in app.blueprints:
app.register_blueprint(api_v3, url_prefix="/api/v3")
return app.test_client()
def _memory(total_mb, used_mb, available_mb):
m = MagicMock()
m.total = total_mb * MB
m.used = used_mb * MB
m.available = available_mb * MB
m.percent = round(used_mb / total_mb * 100, 1)
return m
def _get_status(client, memory):
# The endpoint caches for 10s; bypass so each case is measured fresh.
with patch("web_interface.cache.get_cached", return_value=None), \
patch("psutil.virtual_memory", return_value=memory), \
patch("psutil.cpu_percent", return_value=5.0), \
patch("psutil.boot_time", return_value=0.0):
resp = client.get("/api/v3/system/status")
assert resp.status_code == 200, resp.data
return json.loads(resp.data)["data"]
def test_available_memory_is_reported(client):
data = _get_status(client, _memory(total_mb=905, used_mb=620, available_mb=284))
assert "memory_available_mb" in data
assert data["memory_available_mb"] == pytest.approx(284, abs=0.5)
def test_available_is_not_total_minus_used(client):
# The case the readout exists for: 600MB is "not used", but only 300MB can
# actually be allocated. Reporting used% alone would call this healthy.
data = _get_status(client, _memory(total_mb=1000, used_mb=400, available_mb=300))
derived = data["memory_total_mb"] - data["memory_used_mb"]
assert derived == pytest.approx(600, abs=1)
assert data["memory_available_mb"] == pytest.approx(300, abs=0.5)
assert data["memory_available_mb"] != pytest.approx(derived, abs=1), \
"available must come from MemAvailable, not be derived from used"
def test_existing_memory_fields_are_unchanged(client):
data = _get_status(client, _memory(total_mb=905, used_mb=620, available_mb=284))
assert data["memory_total_mb"] == pytest.approx(905, abs=0.5)
assert data["memory_used_mb"] == pytest.approx(620, abs=0.5)
assert "memory_used_percent" in data
def test_a_nearly_exhausted_board_reports_a_small_number(client):
# 73MB available is what the board actually read when it stopped being able
# to fork. The readout has to surface that rather than round it away.
data = _get_status(client, _memory(total_mb=905, used_mb=800, available_mb=73))
assert data["memory_available_mb"] == pytest.approx(73, abs=0.5)
-99
View File
@@ -1,99 +0,0 @@
"""Frame pacing and FPS health reporting must not depend on the wall clock.
These devices have no RTC, so the system clock jumps by however wrong boot
time was the moment NTP first syncs. The render loop sleeps the *remainder*
of each frame budget:
frame_elapsed = <now> - frame_started
time.sleep(max(0.0, frame_interval - frame_elapsed))
With a wall-clock `now`, a backward jump makes frame_elapsed negative, so
`frame_interval - frame_elapsed` exceeds the whole budget and the render loop
stalls for the size of the correction. A forward jump instead inflates the
p99 and worst-frame numbers the telemetry reports.
"""
import ast
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
COORD = (Path(__file__).resolve().parent.parent
/ "src" / "vegas_mode" / "coordinator.py")
TREE = ast.parse(COORD.read_text(encoding="utf-8"))
def _assignments_of(name):
"""Every `name = <expr>` in the module, as unparsed source."""
out = []
for node in ast.walk(TREE):
if isinstance(node, ast.Assign):
for target in node.targets:
if isinstance(target, ast.Name) and target.id == name:
out.append((node.lineno, ast.unparse(node.value)))
return out
def test_per_frame_timestamps_are_monotonic():
for name in ("frame_started", "frame_elapsed"):
assigns = _assignments_of(name)
assert assigns, f"{name} is no longer assigned -- has the loop changed?"
for lineno, expr in assigns:
assert "time.time()" not in expr, (
f"{name} at line {lineno} uses the wall clock ({expr!r}). A "
"backward NTP step makes the per-frame delta negative and the "
"loop then sleeps longer than the whole frame budget.")
assert "time.monotonic()" in expr, (
f"{name} at line {lineno} is {expr!r}, expected monotonic")
def test_the_fps_window_is_monotonic():
for lineno, expr in _assignments_of("current_time"):
assert "time.monotonic()" in expr, (
f"current_time at line {lineno} is {expr!r}; fps is frames divided "
"by this delta, so a clock step would corrupt the rate itself")
def test_health_state_is_not_reset_every_iteration():
"""run_iteration() runs once per cycle -- locals here reset every few seconds.
As locals, `last_fps_health_log = 0.0` made the 300s heartbeat fire on the
first sample of every iteration, and a recovery spanning two iterations was
never reported because was_degraded had already gone back to False.
"""
run_iteration = next(
(n for n in ast.walk(TREE)
if isinstance(n, ast.FunctionDef) and n.name == "run_iteration"), None)
assert run_iteration is not None, "run_iteration() not found"
local_names = {t.id for n in ast.walk(run_iteration)
if isinstance(n, ast.Assign)
for t in n.targets if isinstance(t, ast.Name)}
for leaked in ("last_fps_health_log", "was_degraded"):
assert leaked not in local_names, (
f"{leaked} is a local of run_iteration() again, so it resets every "
"cycle -- the heartbeat degenerates to once per iteration")
body = ast.unparse(run_iteration)
assert "self._fps_last_health_log" in body and "self._fps_was_degraded" in body, (
"the health state should live on the coordinator, across iterations")
def test_start_clears_stale_health_state():
"""A new run must not inherit "was degraded" from the previous one."""
start = next((n for n in ast.walk(TREE)
if isinstance(n, ast.FunctionDef) and n.name == "start"), None)
assert start is not None, "start() not found"
body = ast.unparse(start)
assert "self._fps_last_health_log" in body and "self._fps_was_degraded" in body, (
"start() does not reset the FPS health state")
def test_the_degraded_threshold_is_documented():
"""The 90% band is deliberate; say so where the constant is defined."""
source = COORD.read_text(encoding="utf-8")
idx = source.index("_FPS_HEALTHY_FRACTION = ")
preamble = source[max(0, idx - 700):idx]
assert "90%" in preamble or "0.9" in preamble, (
"the degradation threshold is not explained at its definition, so "
"'below target' reads as a bug rather than a deliberate band")
@@ -229,44 +229,6 @@ class TestSavePluginConfig:
"REAL-KEY-0123456789", "an unrelated edit destroyed the API key"
assert env.fresh_load()[PLUGIN_ID]["city"] == "Dallas"
def test_an_unrelated_edit_does_not_erase_array_item_secrets(self, env):
"""The scalar api_key case above, but for a list of credentials.
remove_empty_secrets recursed into dicts only, so a list went into
deep_merge untouched -- and lists merge by *replacement*. Saving any
unrelated field posted [{"token": ""}, ...] straight over the stored
array and destroyed every token in it at once.
"""
assert self._save(env, {"accounts": [
{"name": "a", "token": "REAL-A"},
{"name": "b", "token": "REAL-B"},
], "city": "Austin"}).status_code == 200
# the user changes the city; both masked tokens ride along blank
assert self._save(env, {"accounts": [
{"name": "a", "token": ""},
{"name": "b", "token": ""},
], "city": "Dallas"}).status_code == 200
merged = env.fresh_load()[PLUGIN_ID]
assert [a.get("token") for a in merged["accounts"]] == \
["REAL-A", "REAL-B"], "an unrelated edit destroyed the array secrets"
assert [a["name"] for a in merged["accounts"]] == ["a", "b"]
assert merged["city"] == "Dallas"
def test_one_array_secret_can_be_changed_without_losing_the_rest(self, env):
assert self._save(env, {"accounts": [
{"name": "a", "token": "REAL-A"},
{"name": "b", "token": "REAL-B"},
]}).status_code == 200
assert self._save(env, {"accounts": [
{"name": "a", "token": ""},
{"name": "b", "token": "NEW-B"},
]}).status_code == 200
merged = env.fresh_load()[PLUGIN_ID]
assert [a.get("token") for a in merged["accounts"]] == ["REAL-A", "NEW-B"]
def test_a_secret_can_still_be_changed(self, env):
"""Dropping blanks must not stop a real new value from being saved."""
self._save(env, {"api_key": "first-key"})
@@ -1,45 +0,0 @@
"""The validation logging ran before separate_secrets, so it logged credentials.
api_v3's plugin-config save logged `Full config: {plugin_config}` at INFO and
`Config that failed: {plugin_config}` at ERROR. Both run *before*
separate_secrets(), so plugin_config still held the values the user just typed
into the form -- API keys and tokens went to the journal in clear text.
"""
import re
from pathlib import Path
import pytest
SOURCE = (Path(__file__).resolve().parents[2]
/ "web_interface" / "blueprints" / "api_v3.py")
#: Objects that still hold submitted secret values at the point these log
#: calls run. Interpolating one whole into a log message leaks credentials.
UNREDACTED = ("plugin_config", "secrets_config", "current_secrets")
def _logging_lines():
for number, line in enumerate(SOURCE.read_text(encoding="utf-8").splitlines(), 1):
stripped = line.strip()
if stripped.startswith("#"):
continue
if re.match(r"logger\.(debug|info|warning|error|critical|exception)\(", stripped):
yield number, stripped
@pytest.mark.parametrize("name", UNREDACTED)
def test_no_log_call_interpolates_a_whole_secret_bearing_object(name):
# {name} or {name['k']} leaks; {list(name.keys())} and {len(name)} do not.
bare = re.compile(r"\{" + re.escape(name) + r"(\[[^\]]*\])*\}")
offenders = [f"{n}: {text}" for n, text in _logging_lines() if bare.search(text)]
assert not offenders, (
f"{name} still holds submitted secrets where these log calls run:\n "
+ "\n ".join(offenders))
def test_the_guard_would_notice_a_reintroduced_leak():
"""Pin the detector itself, so a rewrite cannot silently stop matching."""
bare = re.compile(r"\{" + re.escape("plugin_config") + r"(\[[^\]]*\])*\}")
assert bare.search('logger.info(f"Full config: {plugin_config}")')
assert bare.search("logger.error(f\"{plugin_config['api_key']}\")")
assert not bare.search('logger.info(f"{list(plugin_config.keys())}")')
-65
View File
@@ -17,7 +17,6 @@ from src.web_interface.secret_helpers import (
separate_secrets,
mask_secret_fields,
mask_all_secret_values,
merge_secrets,
remove_empty_secrets,
)
@@ -240,67 +239,3 @@ class TestRemoveEmptySecrets:
def test_keeps_falsey_non_string_values(self):
# 0 and False are neither None nor blank strings — they are kept.
assert remove_empty_secrets({"a": 0, "b": False}) == {"a": 0, "b": False}
class TestArrayItemSecrets:
"""Lists merge by replacement, so a blanked array wipes stored credentials.
remove_empty_secrets recursed into dicts but let a list through untouched,
so [{"token": ""}] went straight into deep_merge and overwrote the stored
list. Saving any unrelated setting destroyed every token in the array.
"""
STORED = {"accounts": [{"name": "a", "token": "REAL-A"},
{"name": "b", "token": "REAL-B"}]}
def test_an_unrelated_save_keeps_every_stored_token(self):
posted = {"accounts": [{"name": "a", "token": ""},
{"name": "b", "token": ""}]}
merged = merge_secrets(self.STORED, remove_empty_secrets(posted))
assert [a["token"] for a in merged["accounts"]] == ["REAL-A", "REAL-B"]
def test_editing_one_entry_leaves_the_others_alone(self):
posted = {"accounts": [{"name": "a", "token": ""},
{"name": "b", "token": "NEW-B"}]}
merged = merge_secrets(self.STORED, remove_empty_secrets(posted))
assert [a["token"] for a in merged["accounts"]] == ["REAL-A", "NEW-B"]
def test_a_new_entry_is_appended(self):
posted = {"accounts": [{"name": "a", "token": ""},
{"name": "b", "token": ""},
{"name": "c", "token": "NEW-C"}]}
merged = merge_secrets(self.STORED, remove_empty_secrets(posted))
assert [a["token"] for a in merged["accounts"]] == \
["REAL-A", "REAL-B", "NEW-C"]
def test_a_list_of_bare_strings_merges_by_index(self):
merged = merge_secrets({"keys": ["K1", "K2", "K3"]},
remove_empty_secrets({"keys": ["", "K2-NEW", ""]}))
assert merged["keys"] == ["K1", "K2-NEW", "K3"]
def test_an_all_blank_list_is_dropped_entirely(self):
posted = {"accounts": [{"token": ""}, {"token": ""}]}
assert "accounts" not in remove_empty_secrets(posted)
def test_plain_dict_secrets_are_unaffected(self):
merged = merge_secrets({"api_key": "OLD", "other": "keep"},
remove_empty_secrets({"api_key": "", "other": "changed"}))
assert merged == {"api_key": "OLD", "other": "changed"}
def test_a_removed_entry_takes_its_secret_with_it(self):
"""The regular config's list is authoritative about how many items
exist, and the secrets list runs parallel to it -- see
ConfigManager._strip_secrets_recursive. So a shorter incoming list
must shorten the stored secrets too, or the two fall out of step."""
posted = {"accounts": [{"name": "a", "token": "NEW-A"}]}
merged = merge_secrets(self.STORED, remove_empty_secrets(posted))
assert [a["token"] for a in merged["accounts"]] == ["NEW-A"]
def test_an_emptied_item_stays_a_dict_not_none(self):
"""None there stops the list looking parallel, and
_strip_secrets_recursive then drops the whole key from the main
config -- deleting the item's non-secret fields as well."""
pruned = remove_empty_secrets(
{"accounts": [{"token": "real"}, {"token": ""}]})
assert pruned["accounts"] == [{"token": "real"}, {}]
assert None not in pruned["accounts"]
+32 -46
View File
@@ -22,8 +22,7 @@ logger = logging.getLogger(__name__)
from src.web_interface.api_helpers import success_response, error_response, validate_request_json
from src.web_interface.errors import ErrorCode
from src.web_interface.secret_helpers import (find_secret_fields, mask_all_secret_values,
merge_secrets, remove_empty_secrets,
separate_secrets,
remove_empty_secrets, separate_secrets,
strip_masked_values)
from src.web_interface.error_handler import describe_exception, redact_text
from src.plugin_system.operation_types import OperationType
@@ -598,7 +597,7 @@ def save_dim_schedule_config():
dim_brightness = 30
else:
dim_brightness = int(dim_brightness_raw)
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return error_response(
ErrorCode.VALIDATION_ERROR,
"dim_brightness must be an integer between 0 and 100",
@@ -798,7 +797,7 @@ def save_main_config():
}), 400
try:
target_fps = int(raw_target_fps)
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return jsonify({
'status': 'error',
'message': "Invalid value for target_fps: must be an integer"
@@ -868,7 +867,7 @@ def save_main_config():
mux_val = int(data['multiplexing'])
if mux_val < 0 or mux_val > 22:
return jsonify({'status': 'error', 'message': f"Invalid multiplexing value '{data['multiplexing']}'. Must be an integer from 0 to 22."}), 400
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return jsonify({'status': 'error', 'message': f"Invalid multiplexing value '{data['multiplexing']}'. Must be an integer from 0 to 22."}), 400
# Validate pixel_mapper_config (free-form mapper string, e.g. "U-mapper;Rotate:90")
@@ -886,7 +885,7 @@ def save_main_config():
rat_val = int(data['row_address_type'])
if rat_val < 0 or rat_val > 4:
return jsonify({'status': 'error', 'message': f"Invalid row_address_type '{data['row_address_type']}'. Must be an integer from 0 to 4."}), 400
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return jsonify({'status': 'error', 'message': f"Invalid row_address_type '{data['row_address_type']}'. Must be an integer from 0 to 4."}), 400
# Handle hardware settings
@@ -911,7 +910,7 @@ def save_main_config():
if rp1_val not in (0, 1):
return jsonify({'status': 'error', 'message': "rp1_rio must be 0 (PIO) or 1 (RIO)"}), 400
current_config['display']['runtime']['rp1_rio'] = rp1_val
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return jsonify({'status': 'error', 'message': "rp1_rio must be 0 or 1"}), 400
# Handle checkboxes - coerce to bool to ensure proper JSON types
@@ -964,7 +963,7 @@ def save_main_config():
copies = None
try:
copies = int(data['double_sided_copies'])
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
if enabled:
return jsonify({'status': 'error', 'message': "Double-sided copies must be an integer"}), 400
if copies is not None and not (2 <= copies <= 8):
@@ -1037,7 +1036,7 @@ def save_main_config():
if data.get('vegas_extend_threshold_screens') not in ('', None):
try:
screens = float(data['vegas_extend_threshold_screens'])
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return jsonify({
'status': 'error',
'message': "Invalid value for vegas_extend_threshold_screens: "
@@ -1054,7 +1053,7 @@ def save_main_config():
if data.get('vegas_max_plugin_width_ratio') not in ('', None):
try:
ratio = float(data['vegas_max_plugin_width_ratio'])
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return jsonify({
'status': 'error',
'message': "Invalid value for vegas_max_plugin_width_ratio: "
@@ -1102,7 +1101,7 @@ def save_main_config():
continue
try:
int_value = int(raw_value)
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return jsonify({
'status': 'error',
'message': f"Invalid value for {field_name}: must be an integer"
@@ -1154,7 +1153,7 @@ def save_main_config():
if not (1024 <= port_val <= 65535):
return jsonify({'status': 'error', 'message': "sync_port must be between 1024 and 65535"}), 400
current_config['sync']['port'] = port_val
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return jsonify({'status': 'error', 'message': "sync_port must be an integer"}), 400
if "sync_follower_position" in data:
@@ -1198,7 +1197,7 @@ def save_main_config():
raw_value = data.pop(field)
try:
int_value = int(raw_value)
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return jsonify({'status': 'error',
'message': f"Invalid duration for {field}: must be an integer"}), 400
current_config['display']['display_durations'][field] = int_value
@@ -1221,7 +1220,7 @@ def save_main_config():
continue
try:
int_value = int(raw_value)
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return jsonify({'status': 'error',
'message': f"Invalid duration for mode '{mode_key}': must be an integer"}), 400
current_config['display']['display_durations'][mode_key] = int_value
@@ -1297,10 +1296,7 @@ def save_main_config():
if secrets_config:
if plugin_id not in current_secrets:
current_secrets[plugin_id] = {}
# Lists merge by replacement, so deep_merge here wrote a
# blanked array straight over the stored credentials.
current_secrets[plugin_id] = merge_secrets(
current_secrets[plugin_id], secrets_config)
current_secrets[plugin_id] = deep_merge(current_secrets[plugin_id], secrets_config)
# Save secrets file
api_v3.config_manager.save_raw_file_content('secrets', current_secrets)
@@ -1563,11 +1559,6 @@ def get_system_status():
'memory_used_percent': round(memory_percent, 1),
'memory_total_mb': round(memory.total / (1024 * 1024), 1),
'memory_used_mb': round(memory.used / (1024 * 1024), 1),
# MemAvailable, not total-minus-used: it accounts for reclaimable
# page cache, so it is what actually predicts memory trouble. A
# board can read 70% "used" and be fine, or read the same and be
# about to fail fork(), and only this number tells them apart.
'memory_available_mb': round(memory.available / (1024 * 1024), 1),
'cpu_temp': round(cpu_temp, 1) if cpu_temp is not None else None,
'disk_used_percent': round(disk_percent, 1),
'disk_total_gb': round(disk.total / (1024 * 1024 * 1024), 1),
@@ -5127,7 +5118,7 @@ def save_plugin_config():
converted_array.append(int(v))
else:
converted_array.append(float(v))
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
converted_array.append(v)
else:
converted_array.append(v)
@@ -5152,7 +5143,7 @@ def save_plugin_config():
converted_array.append(int(v))
else:
converted_array.append(float(v))
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
converted_array.append(v)
else:
converted_array.append(v)
@@ -5189,7 +5180,7 @@ def save_plugin_config():
converted_array.append(int(v))
else:
converted_array.append(float(v))
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
converted_array.append(v)
else:
converted_array.append(v)
@@ -5213,7 +5204,7 @@ def save_plugin_config():
converted_array.append(int(v))
else:
converted_array.append(float(v))
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
converted_array.append(v)
else:
converted_array.append(v)
@@ -5380,7 +5371,7 @@ def save_plugin_config():
if isinstance(v, str):
try:
converted.append(int(v) if item_type == 'integer' else float(v))
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
converted.append(v)
else:
converted.append(v)
@@ -5505,7 +5496,7 @@ def save_plugin_config():
try:
normalized[key] = int(value_stripped)
continue
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
pass
elif isinstance(value, (int, float)):
normalized[key] = int(value)
@@ -5523,7 +5514,7 @@ def save_plugin_config():
try:
normalized[key] = float(value_stripped)
continue
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
pass
elif isinstance(value, (int, float)):
normalized[key] = float(value)
@@ -5578,7 +5569,7 @@ def save_plugin_config():
try:
normalized_array.append(int(v))
continue
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
pass
elif isinstance(v, (int, float)):
normalized_array.append(int(v))
@@ -5588,7 +5579,7 @@ def save_plugin_config():
try:
normalized_array.append(float(v))
continue
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
pass
elif isinstance(v, (int, float)):
normalized_array.append(float(v))
@@ -5604,7 +5595,7 @@ def save_plugin_config():
if isinstance(v, str):
try:
normalized_array.append(int(v))
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
normalized_array.append(v)
elif isinstance(v, (int, float)):
normalized_array.append(int(v))
@@ -5618,7 +5609,7 @@ def save_plugin_config():
if isinstance(v, str):
try:
normalized_array.append(float(v))
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
normalized_array.append(v)
else:
normalized_array.append(v)
@@ -5641,7 +5632,7 @@ def save_plugin_config():
if isinstance(value, str):
try:
normalized[key] = int(value)
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
normalized[key] = value
else:
normalized[key] = value
@@ -5650,7 +5641,7 @@ def save_plugin_config():
if isinstance(value, str):
try:
normalized[key] = float(value)
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
normalized[key] = value
else:
normalized[key] = value
@@ -5684,10 +5675,8 @@ def save_plugin_config():
if schema:
# Log what we're validating for debugging
logger.info(f"Validating config for {plugin_id}")
# Only the shape. plugin_config still holds the submitted secret
# values at this point -- separate_secrets does not run until
# below -- so logging it wrote live credentials to the journal.
logger.info(f"Config keys being validated: {list(plugin_config.keys())}")
logger.info(f"Full config: {plugin_config}")
# Get enhanced schema keys (including injected core properties)
# We need to create an enhanced schema to get the actual allowed keys
@@ -5710,8 +5699,7 @@ def save_plugin_config():
# Log validation errors for debugging
logger.error(f"Config validation failed for {plugin_id}")
logger.error(f"Validation errors: {validation_errors}")
# Keys only, for the same reason as above.
logger.error(f"Config keys that failed: {list(plugin_config.keys())}")
logger.error(f"Config that failed: {plugin_config}")
logger.error(f"Schema properties: {list(enhanced_schema.get('properties', {}).keys())}")
# Also print to console for immediate visibility
@@ -5762,9 +5750,7 @@ def save_plugin_config():
if secrets_config:
if plugin_id not in current_secrets:
current_secrets[plugin_id] = {}
# See above -- secrets lists must merge element-wise.
current_secrets[plugin_id] = merge_secrets(
current_secrets[plugin_id], secrets_config)
current_secrets[plugin_id] = deep_merge(current_secrets[plugin_id], secrets_config)
# Save secrets file
try:
api_v3.config_manager.save_raw_file_content('secrets', current_secrets)
@@ -6793,7 +6779,7 @@ def get_font_preview() -> tuple[Response, int] | Response:
# Safe integer parsing for size
try:
size = int(request.args.get('size', 12))
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return jsonify({'status': 'error', 'message': 'Invalid font size'}), 400
if not font_filename:
@@ -8374,7 +8360,7 @@ def clear_old_errors():
context={'provided_value': raw_max_age},
status_code=400
)
except (ValueError, TypeError, OverflowError):
except (ValueError, TypeError):
return error_response(
error_code=ErrorCode.INVALID_INPUT,
message="max_age_hours must be a valid integer",
File diff suppressed because it is too large Load Diff
+3
View File
@@ -426,6 +426,9 @@ a, button, input, select, textarea {
.md\:hidden { display: none; }
.md\:block { display: block; }
.md\:w-auto { width: auto; }
/* composer.html labels its toolbar buttons `hidden md:inline`, so without
this the label is hidden at every width and the buttons stay icon-only. */
.md\:inline { display: inline; }
}
@media (min-width: 1024px) {
+1 -11
View File
@@ -126,17 +126,7 @@ window.showRestartPending = function(message) {
} catch { /* private browsing */ }
const banner = document.getElementById('restart-pending-banner');
const text = document.getElementById('restart-pending-text');
if (text) {
// Without the else-branch a config save inherited whatever wording the
// previous update left in the DOM: showRestartPending() clears the
// stored text but used to leave the element itself alone. The default
// is read back from the server-rendered copy rather than duplicated
// here, so the template stays the one place that owns the string.
if (text.dataset.defaultText === undefined) {
text.dataset.defaultText = text.textContent.trim();
}
text.textContent = message || text.dataset.defaultText;
}
if (text && message) text.textContent = message;
if (banner) banner.style.display = 'block';
};
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,804 @@
/**
* ComposerCanvas — stateless LED matrix canvas renderer.
*
* Coordinate system: LED pixels (integers). All drawing multiplies by SCALE.
* PIL draw.text(x,y) is top-left; canvas fillText(x,y) is baseline.
* → Canvas text cy = (actualY + fontSizePx) * SCALE
*
* Anchors: element x/y are offsets from their anchor point:
* xAnchor=null/'left' → x is fixed offset from left
* xAnchor='center' → x is offset from width/2
* xAnchor='right' → x is offset inward from right edge
* yAnchor follows the same pattern with 'top'/'middle'/'bottom'
*
* Breakpoints: elements with minWidth > currentMatrixW are rendered at 25% opacity.
*
* Resize handles: drawn on selected rectangles; 8 handles (corners + edge mids).
*/
window.ComposerCanvas = (() => {
'use strict';
let _canvas = null;
let _ctx = null;
let _showGrid = true;
//: Element types the canvas draws resize handles for. Exported because the
//: editor has to gate its resize and hover behaviour on exactly this list --
//: the two had drifted, so handles appeared on five shapes that could not
//: actually be resized.
const RESIZABLE_TYPES = ['rectangle', 'rounded_rectangle', 'ellipse', 'arc',
'gauge', 'sparkline'];
const DISPLAY_PRESETS = [
{ label: '64×32', w: 64, h: 32 },
{ label: '128×32', w: 128, h: 32 },
{ label: '128×64', w: 128, h: 64 },
{ label: '256×32', w: 256, h: 32 },
{ label: '256×64', w: 256, h: 64 },
];
const FONT_MAP = {
press_start: { family: "'PressStart2P', monospace", sizePx: 8, charW: 8 },
four_by_six: { family: 'monospace', sizePx: 6, charW: 4 },
five_by_seven: { family: 'monospace', sizePx: 7, charW: 5 },
};
const ELEMENT_DEFAULTS = {
text: {
text: 'Hello', font: 'press_start',
r: 255, g: 255, b: 255,
text2: '', lineSpacing: 2, textAlign: 'left',
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
dynamic_text: {
binding: { source: 'config', key: '', format: null },
font: 'press_start', textAlign: 'left',
r: 255, g: 200, b: 100,
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
clock: {
format: '%H:%M', font: 'press_start',
r: 100, g: 255, b: 100,
format2: '', lineSpacing: 2, textAlign: 'left',
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
rectangle: {
width: 20, height: 8,
fillR: 0, fillG: 0, fillB: 128, hasFill: true,
outR: 255, outG: 255, outB: 255, hasOutline: true,
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
ellipse: {
width: 24, height: 12,
fillR: 0, fillG: 100, fillB: 200, hasFill: true,
outR: 100, outG: 180, outB: 255, hasOutline: true,
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
arc: {
width: 24, height: 24,
startAngle: 0, endAngle: 270, lineWidth: 2,
r: 255, g: 200, b: 0,
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
pixel: {
r: 255, g: 255, b: 255,
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
rounded_rectangle: {
width: 24, height: 10, borderRadius: 3,
fillR: 0, fillG: 80, fillB: 180, hasFill: true,
outR: 120, outG: 180, outB: 255, hasOutline: true,
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
line: {
x0: 0, y0: 16, x1: 63, y1: 16,
r: 180, g: 180, b: 180, lineWidth: 1,
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
divider: {
orientation: 'horizontal', y: 16, x: 64,
r: 100, g: 100, b: 100,
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
progress_bar: {
barWidth: 60, barHeight: 6,
binding: { source: 'config', key: '', format: null },
r: 80, g: 200, b: 80,
bgR: 30, bgG: 30, bgB: 30, hasBg: true,
outR: 100, outG: 100, outB: 100, hasOutline: true,
previewPct: 65,
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
countdown: {
binding: { source: 'config', key: '', format: null },
countdownFormat: 'dh',
font: 'four_by_six', textAlign: 'left',
r: 255, g: 180, b: 0,
previewText: '42d 3h',
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
marquee: {
text: 'Scrolling text', font: 'press_start',
r: 255, g: 255, b: 255,
scrollSpeed: 1, gap: 16, direction: 'left',
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
section: {
label: 'Section',
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
pips: {
count: 5, filled: 3, pipSize: 4, pipSpacing: 2,
r: 255, g: 200, b: 0,
emptyR: 50, emptyG: 50, emptyB: 50, showEmpty: true,
binding: { source: 'config', key: '', format: null },
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
sparkline: {
width: 40, height: 12,
barCount: 8, barSpacing: 1,
r: 80, g: 200, b: 120,
bgR: 30, bgG: 30, bgB: 30, hasBg: false,
binding: { source: 'config', key: '', format: null },
previewData: '0.3,0.6,0.4,0.8,0.5,0.9,0.7,0.85',
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
gauge: {
width: 32, height: 32,
startAngle: 135, endAngle: 45, lineWidth: 3,
binding: { source: 'config', key: '', format: null },
r: 80, g: 220, b: 80,
trackR: 40, trackG: 40, trackB: 40, hasTrack: true,
showLabel: true, font: 'four_by_six', labelR: 200, labelG: 200, labelB: 200,
previewPct: 65,
xAnchor: null, yAnchor: null, minWidth: 0, locked: false, blink: false, visible: true,
},
};
// ── Anchor resolution ────────────────────────────────────────────────
function resolveAnchor(val, anchor, dim) {
if (!anchor || anchor === 'left' || anchor === 'top') return val;
if (anchor === 'center' || anchor === 'middle') return Math.floor(dim / 2) + val;
if (anchor === 'right' || anchor === 'bottom') return dim - val;
return val;
}
function computeActualPos(el, matrixW, matrixH) {
const ax = resolveAnchor(el.x ?? el.x0 ?? 0, el.xAnchor, matrixW);
const ay = resolveAnchor(el.y ?? el.y0 ?? 0, el.yAnchor, matrixH);
return { x: ax, y: ay };
}
// ── Bounding box (LED pixel space) ──────────────────────────────────
function getBoundingBox(el, matrixW, matrixH) {
const { x: ax, y: ay } = computeActualPos(el, matrixW, matrixH);
const finfo = FONT_MAP[el.font] || FONT_MAP.press_start;
switch (el.type) {
case 'text': {
const t1 = el.text || '', t2 = el.text2 || '';
const w = Math.max(t1.length, t2.length) * finfo.charW;
const h = t2 ? finfo.sizePx * 2 + (el.lineSpacing ?? 2) : finfo.sizePx;
const bx = el.textAlign === 'center' ? ax - w / 2 : el.textAlign === 'right' ? ax - w : ax;
return { x: bx, y: ay, w, h };
}
case 'dynamic_text': {
const key = el.binding?.key || '?';
const w = (`{${key}}`).length * finfo.charW;
const bx = el.textAlign === 'center' ? ax - w / 2 : el.textAlign === 'right' ? ax - w : ax;
return { x: bx, y: ay, w, h: finfo.sizePx };
}
case 'clock': {
const t1 = el.format || '%H:%M', t2 = el.format2 || '';
const w = Math.max(t1.length, t2.length) * finfo.charW;
const h = t2 ? finfo.sizePx * 2 + (el.lineSpacing ?? 2) : finfo.sizePx;
const bx = el.textAlign === 'center' ? ax - w / 2 : el.textAlign === 'right' ? ax - w : ax;
return { x: bx, y: ay, w, h };
}
case 'countdown': {
const pt = el.previewText || '--d --h';
const w = pt.length * finfo.charW;
const bx = el.textAlign === 'center' ? ax - w / 2 : el.textAlign === 'right' ? ax - w : ax;
return { x: bx, y: ay, w, h: finfo.sizePx };
}
case 'rectangle':
case 'rounded_rectangle':
case 'ellipse':
case 'arc':
return { x: ax, y: ay, w: el.width, h: el.height };
case 'pixel':
return { x: ax, y: ay, w: 1, h: 1 };
case 'line': {
// Same anchor offset the draw path applies, or the hit box sits where
// the line used to be.
const dx = ax - el.x0, dy = ay - el.y0;
return {
x: Math.min(el.x0, el.x1) + dx, y: Math.min(el.y0, el.y1) + dy,
w: Math.max(1, Math.abs(el.x1 - el.x0)),
h: Math.max(1, Math.abs(el.y1 - el.y0)),
};
}
case 'divider':
return el.orientation === 'horizontal'
? { x: 0, y: ay, w: matrixW, h: 1 }
: { x: ax, y: 0, w: 1, h: matrixH };
case 'progress_bar':
return { x: ax, y: ay, w: el.barWidth ?? 60, h: el.barHeight ?? 6 };
case 'marquee': {
const mfinfo = FONT_MAP[el.font] || FONT_MAP.press_start;
return { x: 0, y: ay, w: matrixW, h: mfinfo.sizePx };
}
case 'gauge':
return { x: ax, y: ay, w: el.width ?? 32, h: el.height ?? 32 };
case 'sparkline':
return { x: ax, y: ay, w: el.width ?? 40, h: el.height ?? 12 };
case 'pips': {
const pc = el.count ?? 5, ps = el.pipSize ?? 4, pg = el.pipSpacing ?? 2;
return { x: ax, y: ay, w: pc * ps + (pc - 1) * pg, h: ps };
}
case 'section': {
// Was 0x0, so the element was unselectable except through the 3px
// hit-test padding and drew nothing at all -- a user adding one from
// the palette saw an empty canvas.
// Same font resolution as the draw case below, or the box will not
// match the glyphs: getBoundingBox's shared `finfo` falls back to
// press_start, and a section has no font of its own.
const sinfo = FONT_MAP[el.font] || FONT_MAP.four_by_six;
const label = el.label || 'Section';
return { x: ax, y: ay, w: label.length * sinfo.charW, h: sinfo.sizePx };
}
default:
return { x: ax, y: ay, w: 4, h: 4 };
}
}
// ── Resize handle support ─────────────────────────────────────────────
// Returns 8 handle points for a rectangle in LED pixel space
function _getRectHandles(el, matrixW, matrixH) {
const { x: ax, y: ay } = computeActualPos(el, matrixW, matrixH);
const w = el.width, h = el.height;
const cx = ax + w / 2, cy = ay + h / 2;
return {
nw: { x: ax, y: ay },
n: { x: cx, y: ay },
ne: { x: ax + w, y: ay },
w: { x: ax, y: cy },
e: { x: ax + w, y: cy },
sw: { x: ax, y: ay + h },
s: { x: cx, y: ay + h },
se: { x: ax + w, y: ay + h },
};
}
// Returns the handle direction under LED-space point (lx, ly), or null
function getResizeHandle(el, lx, ly, matrixW, matrixH) {
if (!RESIZABLE_TYPES.includes(el.type)) return null;
const handles = _getRectHandles(el, matrixW, matrixH);
const PAD = 4;
for (const [dir, pt] of Object.entries(handles)) {
if (Math.abs(lx - pt.x) <= PAD && Math.abs(ly - pt.y) <= PAD) return dir;
}
return null;
}
const _HANDLE_CURSORS = {
nw: 'nw-resize', n: 'n-resize', ne: 'ne-resize',
w: 'w-resize', e: 'e-resize',
sw: 'sw-resize', s: 's-resize', se: 'se-resize',
};
function getCursorForHandle(handle) {
return _HANDLE_CURSORS[handle] || 'crosshair';
}
// ── Hit test ─────────────────────────────────────────────────────────
function hitTest(el, lx, ly, matrixW, matrixH) {
const PAD = 3;
const bb = getBoundingBox(el, matrixW, matrixH);
return (
lx >= bb.x - PAD && lx <= bb.x + bb.w + PAD &&
ly >= bb.y - PAD && ly <= bb.y + bb.h + PAD
);
}
// ── Draw a single element ─────────────────────────────────────────────
function _drawElement(ctx, el, SCALE, matrixW, matrixH, opts = {}) {
const s = SCALE;
const { x: ax, y: ay } = computeActualPos(el, matrixW, matrixH);
const belowBreakpoint = el.minWidth > 0 && matrixW < el.minWidth;
const hidden = el.visible === false;
ctx.save();
if (hidden) ctx.globalAlpha = 0.12;
else if (belowBreakpoint) ctx.globalAlpha = 0.25;
// Blink animation: when blinkOff, fully hide blinking elements
if (el.blink) {
if (opts.blinkOff) { ctx.restore(); return; }
ctx.globalAlpha *= 0.55;
}
// Helper: compute draw X for text alignment
const _textX = (text, finfo) => {
const tw = text.length * finfo.charW * s;
if (el.textAlign === 'center') return ax * s - tw / 2;
if (el.textAlign === 'right') return ax * s - tw;
return ax * s;
};
try {
switch (el.type) {
case 'section': {
// A design-time label: it marks a region for the author and is not
// emitted into the generated plugin. There was no case here at all,
// so adding "Section Label" from the palette drew nothing and left
// the user with an apparently broken control.
const sfinfo = FONT_MAP[el.font] || FONT_MAP.four_by_six;
ctx.font = `${sfinfo.sizePx * s}px ${sfinfo.family}`;
ctx.fillStyle = `rgba(${el.r ?? 120},${el.g ?? 120},${el.b ?? 120},0.85)`;
ctx.textBaseline = 'top';
ctx.fillText(el.label || 'Section', ax * s, ay * s);
break;
}
case 'text':
case 'dynamic_text':
case 'clock': {
const finfo = FONT_MAP[el.font] || FONT_MAP.press_start;
const key = el.binding?.key || '?';
const pv = opts.previewValues?.[key];
// Substitute {variable} tokens in text using previewValues
const _subVars = str => (str || '').replace(/\{(\w+)\}/g, (_, k) => {
const v = opts.previewValues?.[k];
return v !== undefined && v !== '' ? String(v) : `{${k}}`;
});
const displayText =
el.type === 'text' ? _subVars(el.text || '')
: el.type === 'clock' ? (el.format || '%H:%M')
: (pv !== undefined && pv !== '' ? String(pv) : `{${key}}`);
ctx.font = `${finfo.sizePx * s}px ${finfo.family}`;
ctx.fillStyle = `rgb(${el.r},${el.g},${el.b})`;
ctx.fillText(displayText, _textX(displayText, finfo), (ay + finfo.sizePx) * s);
// Second line (text and clock)
if (el.type === 'text' && el.text2) {
const t2 = _subVars(el.text2);
const y2 = ay + finfo.sizePx + (el.lineSpacing ?? 2);
ctx.fillText(t2, _textX(t2, finfo), (y2 + finfo.sizePx) * s);
}
if (el.type === 'clock' && el.format2) {
const y2 = ay + finfo.sizePx + (el.lineSpacing ?? 2);
ctx.fillText(el.format2, _textX(el.format2, finfo), (y2 + finfo.sizePx) * s);
}
break;
}
case 'countdown': {
const finfo = FONT_MAP[el.font] || FONT_MAP.press_start;
const t = el.previewText || '--d --h';
ctx.font = `${finfo.sizePx * s}px ${finfo.family}`;
ctx.fillStyle = `rgb(${el.r},${el.g},${el.b})`;
ctx.fillText(t, _textX(t, finfo), (ay + finfo.sizePx) * s);
break;
}
case 'rectangle': {
const rx = ax * s, ry = ay * s;
const rw = el.width * s, rh = el.height * s;
if (el.hasFill) {
ctx.fillStyle = `rgb(${el.fillR},${el.fillG},${el.fillB})`;
ctx.fillRect(rx, ry, rw, rh);
}
if (el.hasOutline) {
ctx.strokeStyle = `rgb(${el.outR},${el.outG},${el.outB})`;
// 1 LED pixel, not 1 canvas pixel: at SCALE>1 an unscaled stroke
// renders thinner than the geometry it outlines.
ctx.lineWidth = s;
ctx.strokeRect(rx, ry, rw, rh);
}
break;
}
case 'ellipse': {
const cx = (ax + el.width / 2) * s;
const cy = (ay + el.height / 2) * s;
const lwPx = s; // 1 LED pixel
// Inset by half the stroke, which straddles the path, so the outline
// stays within the element's bounds.
const rx = Math.max(0, (el.width / 2) * s - (el.hasOutline ? lwPx / 2 : 0));
const ry = Math.max(0, (el.height / 2) * s - (el.hasOutline ? lwPx / 2 : 0));
ctx.beginPath();
ctx.ellipse(cx, cy, rx, ry, 0, 0, Math.PI * 2);
if (el.hasFill) {
ctx.fillStyle = `rgb(${el.fillR},${el.fillG},${el.fillB})`;
ctx.fill();
}
if (el.hasOutline) {
ctx.strokeStyle = `rgb(${el.outR},${el.outG},${el.outB})`;
ctx.lineWidth = lwPx;
ctx.stroke();
}
break;
}
case 'arc': {
const cx = (ax + el.width / 2) * s;
const cy = (ay + el.height / 2) * s;
const lwPx = Math.max(1, el.lineWidth || 2) * s;
const rx = Math.max(0, (el.width / 2) * s - lwPx / 2);
const ry = Math.max(0, (el.height / 2) * s - lwPx / 2);
// PIL: 0°=right, clockwise. Canvas: same with anticlockwise=false
const startRad = (el.startAngle ?? 0) * Math.PI / 180;
const endRad = (el.endAngle ?? 270) * Math.PI / 180;
ctx.beginPath();
ctx.ellipse(cx, cy, rx, ry, 0, startRad, endRad, false);
ctx.strokeStyle = `rgb(${el.r},${el.g},${el.b})`;
ctx.lineWidth = lwPx;
ctx.stroke();
break;
}
case 'pixel': {
ctx.fillStyle = `rgb(${el.r},${el.g},${el.b})`;
ctx.fillRect(ax * s, ay * s, s, s);
break;
}
case 'rounded_rectangle': {
const rx = ax * s, ry = ay * s;
const rw = el.width * s, rh = el.height * s;
const rad = Math.min((el.borderRadius ?? 3) * s, rw / 2, rh / 2);
ctx.beginPath();
ctx.roundRect(rx, ry, rw, rh, rad);
if (el.hasFill) {
ctx.fillStyle = `rgb(${el.fillR},${el.fillG},${el.fillB})`;
ctx.fill();
}
if (el.hasOutline) {
ctx.strokeStyle = `rgb(${el.outR},${el.outG},${el.outB})`;
ctx.lineWidth = s;
ctx.stroke();
}
break;
}
case 'line': {
// ax/ay resolve from el.x0 for a line, so this is the anchor offset.
// Without it a line stayed put while every other type moved.
const dx = ax - el.x0, dy = ay - el.y0;
ctx.strokeStyle = `rgb(${el.r},${el.g},${el.b})`;
ctx.lineWidth = Math.max(1, el.lineWidth || 1) * s;
ctx.beginPath();
ctx.moveTo((el.x0 + dx) * s, (el.y0 + dy) * s);
ctx.lineTo((el.x1 + dx) * s, (el.y1 + dy) * s);
ctx.stroke();
break;
}
case 'divider': {
const isH = (el.orientation || 'horizontal') === 'horizontal';
ctx.strokeStyle = `rgb(${el.r},${el.g},${el.b})`;
ctx.lineWidth = s;
ctx.beginPath();
if (isH) {
ctx.moveTo(0, ay * s + 0.5);
ctx.lineTo(_canvas.width, ay * s + 0.5);
} else {
ctx.moveTo(ax * s + 0.5, 0);
ctx.lineTo(ax * s + 0.5, _canvas.height);
}
ctx.stroke();
break;
}
case 'pips': {
const pipCount = Math.max(1, el.count ?? 5);
const pvPips = opts.previewValues?.[el.binding?.key];
const filledN = pvPips !== undefined
? Math.max(0, Math.min(pipCount, Math.round(parseFloat(pvPips) || 0)))
: Math.max(0, Math.min(pipCount, el.filled ?? 3));
const ps = Math.max(1, el.pipSize ?? 4);
const pg = Math.max(0, el.pipSpacing ?? 2);
for (let i = 0; i < pipCount; i++) {
const isFilled = i < filledN;
if (!isFilled && !el.showEmpty) continue;
ctx.fillStyle = isFilled
? `rgb(${el.r},${el.g},${el.b})`
: `rgb(${el.emptyR ?? 50},${el.emptyG ?? 50},${el.emptyB ?? 50})`;
ctx.fillRect((ax + i * (ps + pg)) * s, ay * s, ps * s, ps * s);
}
break;
}
case 'sparkline': {
const slW = el.width ?? 40, slH = el.height ?? 12;
const count = Math.max(1, el.barCount ?? 8);
const spacing = el.barSpacing ?? 1;
const barW = Math.max(1, Math.floor((slW - spacing * (count - 1)) / count));
const rawVals = (el.previewData || '').split(',')
.map(v => parseFloat(v.trim())).filter(n => !isNaN(n));
while (rawVals.length < count) rawVals.push(0);
const maxV = Math.max(...rawVals.slice(0, count), 0.001);
const rx = ax * s, ry = ay * s;
if (el.hasBg) {
ctx.fillStyle = `rgb(${el.bgR ?? 30},${el.bgG ?? 30},${el.bgB ?? 30})`;
ctx.fillRect(rx, ry, slW * s, slH * s);
}
ctx.fillStyle = `rgb(${el.r},${el.g},${el.b})`;
for (let i = 0; i < count; i++) {
const norm = Math.max(0, Math.min(1, rawVals[i] / maxV));
const barH = Math.max(1, Math.round(slH * norm));
const bx = rx + (barW + spacing) * i * s;
const by = ry + (slH - barH) * s;
ctx.fillRect(bx, by, barW * s, barH * s);
}
break;
}
case 'gauge': {
const gw = (el.width ?? 32), gh = (el.height ?? 32);
const cx = (ax + gw / 2) * s, cy = (ay + gh / 2) * s;
const rx = (gw / 2) * s, ry = (gh / 2) * s;
const lw = Math.max(1, (el.lineWidth ?? 3));
// rx/ry are canvas pixels ((gw/2)*s) but lw is LED pixels, so
// insetting by lw/2 under-corrected by the scale factor while the
// stroke was drawn at lw*s -- the arc spilled outside the element's
// reported bounding box at any SCALE > 1, and the preview stopped
// matching the generated PIL output.
const lwPx = lw * s;
const startDeg = el.startAngle ?? 135;
const endDeg = el.endAngle ?? 45;
// Arc sweep: from startDeg clockwise to endDeg (PIL convention)
const totalSweep = ((endDeg - startDeg) + 360) % 360 || 360;
const pvGauge = opts.previewValues?.[el.binding?.key];
const pct = pvGauge !== undefined
? Math.max(0, Math.min(100, parseFloat(pvGauge) || 0)) / 100
: Math.max(0, Math.min(100, el.previewPct ?? 65)) / 100;
const fillSweep = totalSweep * pct;
const toRad = deg => (deg - 90) * Math.PI / 180; // canvas 0=top, PIL 0=right → offset -90
// Track arc
if (el.hasTrack !== false) {
ctx.beginPath();
ctx.ellipse(cx, cy, rx - lwPx / 2, ry - lwPx / 2, 0, toRad(startDeg), toRad(startDeg + totalSweep), false);
ctx.strokeStyle = `rgb(${el.trackR ?? 40},${el.trackG ?? 40},${el.trackB ?? 40})`;
ctx.lineWidth = lwPx;
ctx.stroke();
}
// Fill arc
if (pct > 0) {
ctx.beginPath();
ctx.ellipse(cx, cy, rx - lwPx / 2, ry - lwPx / 2, 0, toRad(startDeg), toRad(startDeg + fillSweep), false);
ctx.strokeStyle = `rgb(${el.r},${el.g},${el.b})`;
ctx.lineWidth = lwPx;
ctx.stroke();
}
// Centre label
if (el.showLabel) {
const gfinfo = FONT_MAP[el.font || 'four_by_six'] || FONT_MAP.four_by_six;
const labelText = Math.round(pct * 100) + '%';
ctx.font = `${gfinfo.sizePx * s}px ${gfinfo.family}`;
ctx.fillStyle = `rgb(${el.labelR ?? 200},${el.labelG ?? 200},${el.labelB ?? 200})`;
const ltw = ctx.measureText(labelText).width;
ctx.fillText(labelText, cx - ltw / 2, cy + (gfinfo.sizePx * s) / 2);
}
break;
}
case 'marquee': {
const finfo = FONT_MAP[el.font] || FONT_MAP.press_start;
const text = el.text || 'Scrolling text';
const tw = text.length * finfo.charW * s;
const gap = (el.gap ?? 16) * s;
const totalW = tw + gap;
const tick = opts.animTick ?? 0;
const speed = (el.scrollSpeed ?? 1) * 2;
const scrolled = (tick * speed) % totalW;
// left: text enters from right; right: text enters from left
const startX = el.direction === 'right'
? scrolled - tw
: matrixW * s - scrolled;
ctx.font = `${finfo.sizePx * s}px ${finfo.family}`;
ctx.fillStyle = `rgb(${el.r},${el.g},${el.b})`;
// Clip to canvas width so text doesn't bleed outside
ctx.save();
ctx.beginPath();
ctx.rect(0, ay * s - 1, matrixW * s, (finfo.sizePx + 2) * s);
ctx.clip();
for (let i = -1; i <= 2; i++) {
ctx.fillText(text, startX + i * totalW, (ay + finfo.sizePx) * s);
}
ctx.restore();
break;
}
case 'progress_bar': {
const bw = el.barWidth ?? 60, bh = el.barHeight ?? 6;
const pvPb = opts.previewValues?.[el.binding?.key];
const pct = pvPb !== undefined
? Math.max(0, Math.min(100, parseFloat(pvPb) || 0)) / 100
: Math.max(0, Math.min(100, el.previewPct ?? 65)) / 100;
const rx = ax * s, ry = ay * s;
if (el.hasBg) {
ctx.fillStyle = `rgb(${el.bgR ?? 30},${el.bgG ?? 30},${el.bgB ?? 30})`;
ctx.fillRect(rx, ry, bw * s, bh * s);
}
const fillW = Math.max(0, Math.round(bw * pct));
if (fillW > 0) {
ctx.fillStyle = `rgb(${el.r},${el.g},${el.b})`;
ctx.fillRect(rx, ry, fillW * s, bh * s);
}
if (el.hasOutline) {
ctx.strokeStyle = `rgb(${el.outR ?? 100},${el.outG ?? 100},${el.outB ?? 100})`;
ctx.lineWidth = s;
ctx.strokeRect(rx, ry, bw * s, bh * s);
}
break;
}
}
if (belowBreakpoint) {
ctx.globalAlpha = 0.6;
const bb = getBoundingBox(el, matrixW, matrixH);
ctx.font = `${Math.max(8, s * 2)}px monospace`;
ctx.fillStyle = '#facc15';
ctx.fillText(`≥${el.minWidth}px`, bb.x * s, (bb.y + 4) * s);
}
} finally {
ctx.restore();
}
}
// ── Selection indicator ──────────────────────────────────────────────
function _drawSelection(ctx, el, SCALE, matrixW, matrixH) {
const bb = getBoundingBox(el, matrixW, matrixH);
const PAD = 2, s = SCALE;
const rx = bb.x * s - PAD, ry = bb.y * s - PAD;
const rw = bb.w * s + PAD * 2, rh = bb.h * s + PAD * 2;
ctx.save();
ctx.strokeStyle = '#3b82f6';
ctx.lineWidth = 1;
ctx.setLineDash([3, 2]);
ctx.strokeRect(rx, ry, rw, rh);
ctx.setLineDash([]);
if (el.xAnchor || el.yAnchor) {
ctx.font = `${Math.max(7, s)}px sans-serif`;
ctx.fillStyle = '#a78bfa';
const anchorText = [
el.xAnchor ? `x:${el.xAnchor[0]}` : '',
el.yAnchor ? `y:${el.yAnchor[0]}` : '',
].filter(Boolean).join(' ');
if (anchorText) ctx.fillText(anchorText, rx + 1, ry - 2);
}
// Resize handles: on rect, rounded rect, ellipse
if (RESIZABLE_TYPES.includes(el.type)) {
const handles = _getRectHandles(el, matrixW, matrixH);
const HS = 5;
ctx.fillStyle = 'white';
ctx.strokeStyle = '#2563eb';
ctx.lineWidth = 1;
for (const pt of Object.values(handles)) {
const hx = pt.x * s - HS / 2;
const hy = pt.y * s - HS / 2;
ctx.fillRect(hx, hy, HS, HS);
ctx.strokeRect(hx, hy, HS, HS);
}
} else {
// Corner dots for non-rectangle elements
ctx.fillStyle = '#3b82f6';
const HS = 4;
for (const [hx, hy] of [
[rx - HS / 2, ry - HS / 2], [rx + rw - HS / 2, ry - HS / 2],
[rx - HS / 2, ry + rh - HS / 2], [rx + rw - HS / 2, ry + rh - HS / 2],
]) ctx.fillRect(hx, hy, HS, HS);
}
ctx.restore();
}
// ── Dimension tooltip while dragging ─────────────────────────────────
function drawDragTooltip(ctx, el, SCALE, matrixW, matrixH) {
const bb = getBoundingBox(el, matrixW, matrixH);
const label = el.type === 'rectangle'
? `${el.width}×${el.height}`
: `${bb.x},${bb.y}`;
const s = SCALE;
ctx.save();
ctx.font = `${Math.max(9, s * 1.5)}px monospace`;
const tw = ctx.measureText(label).width;
const tx = bb.x * s, ty = (bb.y - 2) * s;
ctx.fillStyle = 'rgba(0,0,0,0.7)';
ctx.fillRect(tx - 2, ty - 10, tw + 4, 12);
ctx.fillStyle = 'white';
ctx.fillText(label, tx, ty);
ctx.restore();
}
// ── Public API ───────────────────────────────────────────────────────
function init(canvasEl) {
_canvas = canvasEl;
_ctx = canvasEl.getContext('2d');
}
function setGrid(show) { _showGrid = show; }
function updateCanvasSize(matrixW, matrixH, SCALE) {
if (!_canvas) return;
_canvas.width = matrixW * SCALE;
_canvas.height = matrixH * SCALE;
}
function render(elements, selectedId, matrixW, matrixH, SCALE, opts = {}) {
if (!_ctx) return;
const cW = matrixW * SCALE, cH = matrixH * SCALE;
const bg = opts.bgColor;
_ctx.fillStyle = bg ? `rgb(${bg.r},${bg.g},${bg.b})` : '#000';
_ctx.fillRect(0, 0, cW, cH);
if (_showGrid) {
_ctx.strokeStyle = 'rgba(255,255,255,0.07)';
_ctx.lineWidth = 0.5;
for (let x = SCALE; x < cW; x += SCALE) {
_ctx.beginPath(); _ctx.moveTo(x, 0); _ctx.lineTo(x, cH); _ctx.stroke();
}
for (let y = SCALE; y < cH; y += SCALE) {
_ctx.beginPath(); _ctx.moveTo(0, y); _ctx.lineTo(cW, y); _ctx.stroke();
}
}
for (const el of elements) _drawElement(_ctx, el, SCALE, matrixW, matrixH, opts);
if (opts.showRuler) {
_ctx.save();
_ctx.fillStyle = 'rgba(255,255,255,0.08)';
_ctx.fillRect(0, 0, cW, SCALE); // top strip
_ctx.fillRect(0, 0, SCALE, cH); // left strip
_ctx.strokeStyle = 'rgba(255,255,255,0.5)';
_ctx.fillStyle = 'rgba(255,255,255,0.6)';
_ctx.font = `${Math.max(5, SCALE - 1)}px monospace`;
const step = SCALE >= 4 ? 8 : 16;
for (let px = 0; px <= matrixW; px += step) {
const cx = px * SCALE;
const major = px % 32 === 0;
_ctx.lineWidth = 0.5;
_ctx.beginPath(); _ctx.moveTo(cx, 0); _ctx.lineTo(cx, major ? SCALE : SCALE * 0.5); _ctx.stroke();
if (major && px > 0 && px < matrixW - 4) _ctx.fillText(String(px), cx + 1, SCALE - 1);
}
for (let py = 0; py <= matrixH; py += step) {
const cy = py * SCALE;
const major = py % 32 === 0;
_ctx.beginPath(); _ctx.moveTo(0, cy); _ctx.lineTo(major ? SCALE : SCALE * 0.5, cy); _ctx.stroke();
if (major && py > 0 && py < matrixH - 4) _ctx.fillText(String(py), 1, cy + SCALE - 1);
}
_ctx.restore();
}
if (opts.showGuides) {
_ctx.save();
_ctx.strokeStyle = 'rgba(255,60,60,0.45)';
_ctx.lineWidth = 1;
_ctx.setLineDash([4, 3]);
const mx = Math.floor(cW / 2) + 0.5;
const my = Math.floor(cH / 2) + 0.5;
_ctx.beginPath(); _ctx.moveTo(mx, 0); _ctx.lineTo(mx, cH); _ctx.stroke();
_ctx.beginPath(); _ctx.moveTo(0, my); _ctx.lineTo(cW, my); _ctx.stroke();
_ctx.setLineDash([]);
_ctx.restore();
}
const sel = selectedId != null ? elements.find(e => e.id === selectedId) : null;
if (sel) {
_drawSelection(_ctx, sel, SCALE, matrixW, matrixH);
if (opts.showTooltip) drawDragTooltip(_ctx, sel, SCALE, matrixW, matrixH);
}
}
return {
init, render, setGrid, updateCanvasSize,
hitTest, getBoundingBox, computeActualPos, resolveAnchor,
getResizeHandle, getCursorForHandle,
ELEMENT_DEFAULTS, FONT_MAP, DISPLAY_PRESETS, RESIZABLE_TYPES,
};
})();
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,300 @@
"""
{{ plugin_name }} — LEDMatrix Plugin
Generated by LEDMatrix Plugin Composer on {{ generated_date }}
Extension points:
update() → add HTTP/MQTT data-fetching logic here
_get_display_values() → map fetched data to display strings
display() → add new elements or adapt layout per display size
"""
from src.plugin_system.base_plugin import BasePlugin
{% if has_clock or has_countdown %}
from datetime import datetime
{% endif %}
{% if has_blink %}
import time
{% endif %}
{% if has_text_template %}
from collections import defaultdict
{% endif %}
class {{ class_name }}(BasePlugin):
def __init__(self, plugin_id, config, display_manager, cache_manager, plugin_manager):
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
{% for var in config_vars %}
self.{{ var.key }} = config.get({{ var.key | tojson }}, {{ var.default | tojson }})
{% endfor %}
# Live data cache — populated by update(); always {} in static layouts
self._data = {}
def update(self):
"""Fetch and refresh display data.
For dynamic plugins: fetch from APIs/MQTT here and store in self._data.
_get_display_values() will read self._data to produce display strings.
"""
# --- Data sources (add fetch logic here for dynamic plugins) ---
pass
def _get_display_values(self):
"""Map config variables and live data to display-ready strings.
This is the single extension point for v2 data sources:
add self._data lookups here once update() populates them.
"""
return {
{% for var in config_vars %}
{{ var.key | tojson }}: str(self.{{ var.key }}),
{% endfor %}
}
def display(self, force_clear=False):
try:
{% if has_text_template %}
values = defaultdict(str, self._get_display_values())
{% else %}
values = self._get_display_values()
{% endif %}
if force_clear:
self.display_manager.clear()
width = self.display_manager.width
height = self.display_manager.height
{% if bg_color %}
self.display_manager.draw.rectangle([0, 0, width, height], fill={{ bg_color }})
{% endif %}
# ── Elements (rendered bottom to top) ──────────────────────────
{% for el in elements %}
{% set p = " " if el.min_width > 0 else " " %}
{% set pi = (p + " ") if el.blink else p %}
{% if el.min_width > 0 %}
if width >= {{ el.min_width }}: # breakpoint: {{ el.min_width }}px+ displays only
{% endif %}
{% if el.blink %}
{{ p }}if int(time.time() * 2) % 2:
{% endif %}
{% if el.type == 'text' %}
{{ pi }}self.display_manager.draw_text(
{% if el.text_is_template %}
{{ pi }} {{ el.text | tojson }}.format_map(values),
{% else %}
{{ pi }} {{ el.text | tojson }},
{% endif %}
{{ pi }} x={{ el.x_expr }}, y={{ el.y_expr }},
{{ pi }} color={{ el.rgb_tuple }},
{{ pi }} font=self.display_manager.{{ el.font_attr }},
{{ pi }})
{% if el.text2 %}
{{ pi }}self.display_manager.draw_text(
{% if el.text_is_template %}
{{ pi }} {{ el.text2 | tojson }}.format_map(values),
{% else %}
{{ pi }} {{ el.text2 | tojson }},
{% endif %}
{{ pi }} x={{ el.x2_expr }}, y={{ el.y2_expr }},
{{ pi }} color={{ el.rgb_tuple }},
{{ pi }} font=self.display_manager.{{ el.font_attr }},
{{ pi }})
{% endif %}
{% elif el.type == 'dynamic_text' %}
{% if el.binding_source == 'config' %}
{{ pi }}self.display_manager.draw_text(
{{ pi }} values.get({{ el.binding_key | tojson }}, ''),
{{ pi }} x={{ el.x_expr }}, y={{ el.y_expr }},
{{ pi }} color={{ el.rgb_tuple }},
{{ pi }} font=self.display_manager.{{ el.font_attr }},
{{ pi }})
{% endif %}
{% elif el.type == 'clock' %}
{{ pi }}self.display_manager.draw_text(
{{ pi }} datetime.now().strftime({{ el.format | tojson }}),
{{ pi }} x={{ el.x_expr }}, y={{ el.y_expr }},
{{ pi }} color={{ el.rgb_tuple }},
{{ pi }} font=self.display_manager.{{ el.font_attr }},
{{ pi }})
{% if el.format2 %}
{{ pi }}self.display_manager.draw_text(
{{ pi }} datetime.now().strftime({{ el.format2 | tojson }}),
{{ pi }} x={{ el.x2_expr }}, y={{ el.y2_expr }},
{{ pi }} color={{ el.rgb_tuple }},
{{ pi }} font=self.display_manager.{{ el.font_attr }},
{{ pi }})
{% endif %}
{% elif el.type == 'countdown' %}
{{ pi }}_cd_target = float(values.get({{ el.binding_key | tojson }}, 0) or 0)
{{ pi }}_cd_secs = max(0.0, _cd_target - datetime.now().timestamp())
{% if el.countdown_format == 'dhms' %}
{{ pi }}_cd_d, _cd_rem = divmod(int(_cd_secs), 86400)
{{ pi }}_cd_h, _cd_rem = divmod(_cd_rem, 3600)
{{ pi }}_cd_m, _cd_s = divmod(_cd_rem, 60)
{{ pi }}_cd_str = f'{_cd_d}d {_cd_h:02d}:{_cd_m:02d}:{_cd_s:02d}'
{% elif el.countdown_format == 'hms' %}
{{ pi }}_cd_h, _cd_rem = divmod(int(_cd_secs), 3600)
{{ pi }}_cd_m, _cd_s = divmod(_cd_rem, 60)
{{ pi }}_cd_str = f'{_cd_h}h {_cd_m:02d}:{_cd_s:02d}'
{% elif el.countdown_format == 'dhm' %}
{{ pi }}_cd_d, _cd_rem = divmod(int(_cd_secs), 86400)
{{ pi }}_cd_h, _cd_m = divmod(_cd_rem // 60, 60)
{{ pi }}_cd_str = f'{_cd_d}d {_cd_h:02d}h {_cd_m:02d}m'
{% else %}
{{ pi }}_cd_d, _cd_rem = divmod(int(_cd_secs), 86400)
{{ pi }}_cd_h = _cd_rem // 3600
{{ pi }}_cd_str = f'{_cd_d}d {_cd_h}h'
{% endif %}
{{ pi }}self.display_manager.draw_text(
{{ pi }} _cd_str,
{{ pi }} x={{ el.x_expr }}, y={{ el.y_expr }},
{{ pi }} color={{ el.rgb_tuple }},
{{ pi }} font=self.display_manager.{{ el.font_attr }},
{{ pi }})
{% elif el.type == 'rectangle' %}
{{ pi }}self.display_manager.draw.rectangle(
{{ pi }} [{{ el.x_expr }}, {{ el.y_expr }}, {{ el.x2_expr }}, {{ el.y2_expr }}],
{{ pi }} fill={{ el.fill_tuple }},
{{ pi }} outline={{ el.outline_tuple }},
{{ pi }})
{% elif el.type == 'arc' %}
{{ pi }}self.display_manager.draw.arc(
{{ pi }} [{{ el.x_expr }}, {{ el.y_expr }}, {{ el.x2_expr }}, {{ el.y2_expr }}],
{{ pi }} start={{ el.start_angle }}, end={{ el.end_angle }},
{{ pi }} fill={{ el.rgb_tuple }},
{{ pi }} width={{ el.line_width }},
{{ pi }})
{% elif el.type == 'ellipse' %}
{{ pi }}self.display_manager.draw.ellipse(
{{ pi }} [{{ el.x_expr }}, {{ el.y_expr }}, {{ el.x2_expr }}, {{ el.y2_expr }}],
{{ pi }} fill={{ el.fill_tuple }},
{{ pi }} outline={{ el.outline_tuple }},
{{ pi }})
{% elif el.type == 'pixel' %}
{{ pi }}self.display_manager.draw.point(
{{ pi }} [{{ el.x_expr }}, {{ el.y_expr }}],
{{ pi }} fill={{ el.rgb_tuple }},
{{ pi }})
{% elif el.type == 'rounded_rectangle' %}
{{ pi }}self.display_manager.draw.rounded_rectangle(
{{ pi }} [{{ el.x_expr }}, {{ el.y_expr }}, {{ el.x2_expr }}, {{ el.y2_expr }}],
{{ pi }} radius={{ el.border_radius }},
{{ pi }} fill={{ el.fill_tuple }},
{{ pi }} outline={{ el.outline_tuple }},
{{ pi }})
{% elif el.type in ('line', 'divider') %}
{{ pi }}self.display_manager.draw.line(
{{ pi }} [{{ el.x0_expr }}, {{ el.y0_expr }}, {{ el.x1_expr }}, {{ el.y1_expr }}],
{{ pi }} fill={{ el.rgb_tuple }},
{{ pi }} width={{ el.line_width }},
{{ pi }})
{% elif el.type == 'pips' %}
{{ pi }}_pip_filled = max(0, min({{ el.pip_count }}, int(float(values.get({{ el.binding_key | tojson }}, 0) or 0))))
{{ pi }}for _pip_i in range({{ el.pip_count }}):
{{ pi }} _pip_x = ({{ el.x_expr }}) + _pip_i * ({{ el.pip_size }} + {{ el.pip_spacing }})
{{ pi }} _pip_color = {{ el.fill_tuple }} if _pip_i < _pip_filled else {{ el.empty_tuple }}
{% if not el.show_empty %}
{{ pi }} if _pip_i >= _pip_filled:
{{ pi }} continue
{% endif %}
{{ pi }} self.display_manager.draw.rectangle(
{{ pi }} [_pip_x, {{ el.y_expr }}, _pip_x + {{ el.pip_size }} - 1, ({{ el.y_expr }}) + {{ el.pip_size }} - 1],
{{ pi }} fill=_pip_color,
{{ pi }} )
{% elif el.type == 'sparkline' %}
{{ pi }}_sl_raw = str(values.get({{ el.binding_key | tojson }}, '') or '')
{{ pi }}_sl_vals = [float(v.strip()) for v in _sl_raw.split(',') if v.strip()][:{{ el.bar_count }}]
{{ pi }}_sl_vals += [0.0] * max(0, {{ el.bar_count }} - len(_sl_vals))
{{ pi }}_sl_max = max(_sl_vals) if any(_sl_vals) else 1.0
{{ pi }}_sl_bw = max(1, ({{ el.bar_width_px }} - {{ el.bar_spacing }} * ({{ el.bar_count }} - 1)) // {{ el.bar_count }})
{% if el.bg_tuple != 'None' %}
{{ pi }}self.display_manager.draw.rectangle(
{{ pi }} [{{ el.x_expr }}, {{ el.y_expr }}, ({{ el.x_expr }}) + {{ el.bar_width_px }}, ({{ el.y_expr }}) + {{ el.bar_height_px }}],
{{ pi }} fill={{ el.bg_tuple }},
{{ pi }})
{% endif %}
{{ pi }}for _sl_i, _sl_v in enumerate(_sl_vals):
{{ pi }} _sl_norm = max(0.0, min(1.0, _sl_v / (_sl_max or 1)))
{{ pi }} _sl_bh = max(1, round({{ el.bar_height_px }} * _sl_norm))
{{ pi }} _sl_bx = ({{ el.x_expr }}) + (_sl_bw + {{ el.bar_spacing }}) * _sl_i
{{ pi }} _sl_by = ({{ el.y_expr }}) + {{ el.bar_height_px }} - _sl_bh
{{ pi }} self.display_manager.draw.rectangle(
{{ pi }} [_sl_bx, _sl_by, _sl_bx + _sl_bw - 1, _sl_by + _sl_bh - 1],
{{ pi }} fill={{ el.fill_tuple }},
{{ pi }} )
{% elif el.type == 'gauge' %}
{{ pi }}_gv = max(0.0, min(100.0, float(values.get({{ el.binding_key | tojson }}, 0) or 0)))
{{ pi }}_g_total = (({{ el.end_angle }} - {{ el.start_angle }}) % 360) or 360
{{ pi }}_g_sweep = _g_total * _gv / 100.0
{% if el.track_tuple != 'None' %}
{{ pi }}self.display_manager.draw.arc(
{{ pi }} [{{ el.x_expr }}, {{ el.y_expr }}, {{ el.x2_expr }}, {{ el.y2_expr }}],
{{ pi }} start={{ el.start_angle }}, end={{ el.start_angle }} + _g_total,
{{ pi }} fill={{ el.track_tuple }},
{{ pi }} width={{ el.line_width }},
{{ pi }})
{% endif %}
{{ pi }}if _g_sweep > 0:
{{ pi }} self.display_manager.draw.arc(
{{ pi }} [{{ el.x_expr }}, {{ el.y_expr }}, {{ el.x2_expr }}, {{ el.y2_expr }}],
{{ pi }} start={{ el.start_angle }}, end={{ el.start_angle }} + _g_sweep,
{{ pi }} fill={{ el.rgb_tuple }},
{{ pi }} width={{ el.line_width }},
{{ pi }} )
{% if el.show_label %}
{{ pi }}_g_cx = ({{ el.x_expr }}) + ({{ el.x2_expr }} - ({{ el.x_expr }})) // 2
{{ pi }}_g_cy = ({{ el.y_expr }}) + ({{ el.y2_expr }} - ({{ el.y_expr }})) // 2
{{ pi }}self.display_manager.draw_text(
{{ pi }} f'{int(_gv)}%',
{{ pi }} x=_g_cx, y=_g_cy,
{{ pi }} color={{ el.label_tuple }},
{{ pi }} font=self.display_manager.{{ el.font_attr }},
{{ pi }})
{% endif %}
{% elif el.type == 'marquee' %}
{{ pi }}_{{ el.data_key }}_text = {{ el.text | tojson }}
{{ pi }}_{{ el.data_key }}_tw = len(_{{ el.data_key }}_text) * {{ el.char_w }}
{{ pi }}_{{ el.data_key }}_x = int(self._data.get({{ el.data_key | tojson }}, width))
{% if el.direction == 'right' %}
{{ pi }}_{{ el.data_key }}_x += {{ el.scroll_speed }}
{{ pi }}if _{{ el.data_key }}_x > width:
{{ pi }} _{{ el.data_key }}_x = -(_{{ el.data_key }}_tw + {{ el.gap }})
{% else %}
{{ pi }}_{{ el.data_key }}_x -= {{ el.scroll_speed }}
{{ pi }}if _{{ el.data_key }}_x < -(_{{ el.data_key }}_tw + {{ el.gap }}):
{{ pi }} _{{ el.data_key }}_x = width
{% endif %}
{{ pi }}self._data[{{ el.data_key | tojson }}] = _{{ el.data_key }}_x
{{ pi }}self.display_manager.draw_text(
{{ pi }} _{{ el.data_key }}_text,
{{ pi }} x=_{{ el.data_key }}_x, y={{ el.y_expr }},
{{ pi }} color={{ el.rgb_tuple }},
{{ pi }} font=self.display_manager.{{ el.font_attr }},
{{ pi }})
{% elif el.type == 'progress_bar' %}
{{ pi }}_pb_x = {{ el.x_expr }}
{{ pi }}_pb_y = {{ el.y_expr }}
{{ pi }}_pb_pct = max(0.0, min(100.0, float(values.get({{ el.binding_key | tojson }}, 0) or 0))) / 100.0
{{ pi }}_pb_fill_w = int({{ el.bar_width }} * _pb_pct)
{{ pi }}self.display_manager.draw.rectangle(
{{ pi }} [_pb_x, _pb_y, _pb_x + {{ el.bar_width }}, _pb_y + {{ el.bar_height }}],
{{ pi }} fill={{ el.bg_tuple }},
{{ pi }} outline={{ el.outline_tuple }},
{{ pi }})
{{ pi }}if _pb_fill_w > 0:
{{ pi }} self.display_manager.draw.rectangle(
{{ pi }} [_pb_x, _pb_y, _pb_x + _pb_fill_w, _pb_y + {{ el.bar_height }}],
{{ pi }} fill={{ el.fill_tuple }},
{{ pi }} )
{% else %}
{{ pi }}pass # element type "{{ el.type }}" draws nothing
{% endif %}
{% endfor %}
# ── End elements ───────────────────────────────────────────────
self.display_manager.update_display()
except Exception as e:
self.logger.error('Display error: %s', e, exc_info=True)
@@ -687,35 +687,12 @@
const mUsedGb = d.memory_used_mb != null ? (d.memory_used_mb / 1024).toFixed(1) : null;
const mTotGb = d.memory_total_mb != null ? (d.memory_total_mb / 1024).toFixed(1) : null;
const temp = d.cpu_temp != null ? d.cpu_temp + '°C' : 'N/A';
// Available memory is the number that predicts trouble. When it
// runs out the board does not fail cleanly: fork() starts
// returning ENOMEM, so sshd cannot spawn a session and systemd
// cannot respawn the display, while the kernel keeps answering
// pings. Thresholds are drawn from that failure -- it was
// measured at 73MB free, and healthy running sits well above.
// Round ONCE, then colour and label off the same number.
// The API sends one decimal place, so classifying the raw
// value and displaying the rounded one disagreed at the
// boundaries: 149.6 rendered as "150 MB" in red, and 299.6 as
// "300 MB" in amber, both contradicting the threshold the
// colour claims to apply. Which side of the line a spare
// 0.4MB falls on does not matter; the tile agreeing with
// itself does.
const availMb = d.memory_available_mb == null
? null : Math.round(d.memory_available_mb);
const availColor = availMb == null ? 'text-gray-400'
: availMb < 150 ? 'text-red-600'
: availMb < 300 ? 'text-amber-500'
: 'text-green-600';
panel.innerHTML =
diagTile('fa-microchip', 'text-blue-600', 'CPU Usage',
(d.cpu_percent != null ? d.cpu_percent : '--') + '%', null) +
diagTile('fa-memory', 'text-green-600', 'Memory',
(d.memory_used_percent != null ? d.memory_used_percent : '--') + '%',
(mUsedGb && mTotGb) ? `${mUsedGb} / ${mTotGb} GB` : null) +
diagTile('fa-memory', availColor, 'Available Memory',
availMb != null ? `${availMb} MB` : '--',
mTotGb ? `of ${mTotGb} GB total` : null) +
diagTile('fa-thermometer-half', 'text-red-600', 'CPU Temp', temp, null) +
diagTile('fa-hdd', 'text-indigo-600', 'Disk',
(d.disk_used_percent != null ? d.disk_used_percent : '--') + '%',