Files
LEDMatrix/docs/ADAPTIVE_LAYOUT.md
7f7f0d6464 feat: adaptive layout system — size-aware regions, crisp font ladders, image fitting (#393)
* feat(layout): adaptive layout & font scaling system for plugins

Add src/adaptive_layout.py — opt-in core helpers so plugins render
legibly on any panel size without hand-tuned per-display layouts:

- Region: integer rect algebra (bands/columns/weighted splits/centering)
  that partitions space so text bands can't overlap by construction
- Font ladders: ordered (family, size) steps known to render crisply
  (LADDER_GRID: X11 BDFs at native sizes; LADDER_ARCADE: PressStart2P at
  8px multiples) — fitting walks the ladder instead of scaling pixel
  fonts fractionally
- LayoutContext: breakpoint tiers, geometry scale vs. a declared design
  size, and cached fit_text/fit_lines/font_for_rows queries

Generalizes the three patterns proven in the field: f1-scoreboard's
scale factor, masters-tournament's tiers, baseball-scoreboard's font
fallback ladder.

Wiring: BasePlugin gains a lazy .layout property and draw_fit();
FontManager gains get_native_bdf_size() and a cache_generation counter;
manifest schema gains display.design_size and requires.display_size
max_width/max_height; 96x48 joins DEFAULT_TEST_SIZES; the bounds-check
harness records negative-coordinate draws; TextHelper's broken
measurement helpers are fixed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): adaptive image fitting + composite region helpers

Add src/adaptive_images.py — the image counterpart to fit_text:
- fit_image(img, box, mode=contain|cover|fill_height|stretch,
  crop_to_ink, anchor, resample, upscale) promoting the proven plugin
  patterns (football's crop-to-ink fill-height logos, masters' cover
  crop + NEAREST flags, static-image's letterbox). Upscales by default —
  thumbnail()'s downscale-only behavior is why imagery stays tiny on
  big panels.
- draw_fitted_image() pastes aligned within a Region with alpha mask.
- One central Pillow>=9.1 RESAMPLE shim replacing ~15 plugin copies.

LayoutContext.fit_image() caches results per (identity, box size,
options) with a 64-entry LRU; id()-keyed entries pin the source image.
BasePlugin.draw_image() is the one-liner adoption path beside draw_fit.

Composites in adaptive_layout.py: Region.offset() (user x/y-offset
passthrough), scoreboard_regions() (the two-logos-plus-score card math
duplicated across six sports plugins, logo_slot = min(H, W//2)), and
media_row() (art-left/text-right).

Fix LogoHelper's size-blind cache key (stale sizes on panel change);
deprecation note on dead image_utils.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(harness): scale-up fill check, config variants, multi-size dev gallery

Quality gates for adaptive layout:

- fill_metrics()/check_scale_up() in the safety harness: overflow catches
  content too big for a panel, but nothing caught content that stays tiny
  on panels >= 2x the plugin's declared design size. The check measures
  lit-content extents and warns (or fails, when a plugin opts into
  "fill_check": "strict" in test/harness.json) below 50% coverage on the
  doubled axis. Warn-only by default so no existing plugin breaks.

- harness.json "variants": extra runs with config overlays and their own
  golden dirs, so an opt-in mode (e.g. layout_mode: adaptive) is golden-
  tested beside the classic default. check_plugin.py loops base + variants
  and labels variant results mode@name.

- Dev preview server: GET /api/sizes (harness size sample), POST
  /api/render-matrix (render at up to 12 sizes in one call), size-preset
  dropdown, and an "All Sizes" side-by-side gallery in the preview UI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(plugins): adaptive-lib discoverability + advisory version compat warning

Discoverability: re-export the adaptive layout/image API from src.common
(the blessed-helpers package plugin authors already know) — canonical
paths stay src.adaptive_layout / src.adaptive_images so nothing breaks.
Document it in src/common/README.md and cross-link ADAPTIVE_LAYOUT.md
from the developer docs authors actually read (quick reference, API
reference, advanced dev, font manager, dev preview, plugin dev guide);
ADAPTIVE_LAYOUT.md gains adaptive-images, composite-layouts and
preserving-user-customization sections.

Compat: PluginLoader now logs one advisory warning (never raises) when a
plugin's manifest declares a min LEDMatrix version newer than the running
core, checking the min_ledmatrix_version / requires.* / versions[]
spellings found in the wild. Guarded against stale core version numbers.

src/__init__.py __version__ bumped 1.0.0 -> 3.1.0 to match the latest
release tag (v3.1.0) — it had never been updated and the compat check
needs a truthful number. NOTE: verify this matches the intended release
numbering before the next tag.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): add measure_font_crispness — verify a ladder rung isn't blurry

PIL antialiases TTF outlines by default; a 'pixel-style' font only
rasterizes without antialiasing at specific sizes (for PressStart2P:
exact multiples of its 8px design grid). A ladder rung at an unverified
size silently renders blurry on an LED panel — this exact bug shipped in
both text-display's and football-scoreboard's custom TTF ladders
(non-8-multiple PressStart2P sizes, and '5by7.regular'/'4x6-font' at
sizes that were never actually crisp).

measure_font_crispness(font, sample_text) renders the sample and reports
the fraction of ink-bbox pixels that are neither pure black nor pure
white. BDF fonts (real bitmaps) always score 0.0; TTF ladders should be
verified against this before shipping — see the new
TestFontFitting::test_ladder_arcade_is_crisp pattern.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): add fit_text_proportional — proportional sizing vs. always-maximize

fit_text always picks the largest ladder rung that fits its box. That's
right when an element owns dedicated space, but wrong when several
independently-fitted elements need to stay visually harmonious as the
panel grows: a score's box might have generous room while a neighboring
logo scales by a fixed geometry factor via px() — fit_text lets the score
balloon out of proportion (even overlapping the logo) even though its
individual pick is technically correct.

fit_text_proportional(text, box, base_size_px, ladder) instead targets
base_size_px * self.scale (the same scale factor px() already uses),
picking the nearest ladder rung at or below that target, still capped to
what fits the box, floored at the smallest rung when the target is below
every rung. Refactored the shared largest-that-fits/ellipsize walk into
_walk_ladder() so fit_text and fit_text_proportional don't duplicate it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): fit_text_proportional gains an axis-specific scale override

self.scale (min(width_ratio, height_ratio)) is the right conservative
default for anything whose aspect ratio matters, but a caller whose
surrounding composition already scales along a single axis — e.g.
football-scoreboard's logo_slot = min(height, width // 2), which tracks
height alone — needs text sized the same way, or it reads as
under-scaled next to logos that grew on a panel that only got taller
(128x32 -> 128x64: self.scale stays 1.0 since width didn't grow, but
logos still double).

fit_text_proportional(..., scale=None) now accepts an explicit override;
None keeps the existing self.scale default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(layout): scoreboard_regions reserves real center space at 2:1 aspect ratios

logo_slot = min(height, width // 2) has a blind spot: at exactly 2:1
aspect ratio (width == 2 * height -- a very common shape: two, four, or
more square modules stacked into a taller panel) width // 2 and height
are equal, so the two logo slots claim the ENTIRE width and leave zero
pixels for a center column, no matter how large the panel gets. Not a
'small panel' problem -- 96x48, 128x64, and 256x128 (all exactly 2:1) hit
it identically, while the 128x32 design baseline and panels like 192x48
or 256x32 never do, because height is already the tighter constraint
there.

Two new parameters fix it in the one shared helper every scoreboard-style
plugin composes through:

- min_center_fraction / min_center_design_px reserve at least
  max(width * fraction, design_px * ctx.scale) for the center column,
  capping logo_slot further when needed. The scaled design-px term
  matters on small panels where a flat fraction alone reserves too little
  absolute space.
- score_bleed_fraction extends the score's own fit box (not the logo
  slots themselves) a controlled amount into each side -- the same way
  real broadcast scoreboards let a big score number's edges cross into
  the team marks flanking it. Without this the reserve alone can still be
  too narrow for a short score to render without truncating.

score_area is now genuinely narrower than the full card width (previously
identical to status_band/detail_band, which still span the full width and
overlay the logos -- short text there was never the problem).

Verified against the full harness size spread: a real game score like
'17-21' never needs ellipsis at any tested 2:1-or-tighter aspect ratio
(test_score_never_needs_ellipsis_for_a_short_score), and wide panels
(128x32/192x48/256x32-style) are provably unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: document scoreboard_regions' center-reserve and score-bleed params

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: address CodeRabbit review on PR #393

- docs: scope the self.layout note to BasePlugin subclasses (others build
  a LayoutContext directly) and make explicit that adaptive layout is
  opt-in — classic rendering stays unless a plugin adopts the APIs.
- dev_server: broaden the render-request catch (a bad manifest.json now
  returns a clean 400 instead of an unhandled 500) and stop echoing raw
  exception text in the loader-failure responses — full tracebacks go to
  the dev server's console log instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): allowlist plugin_id before any path lookup

CodeQL (py/path-injection): plugin_id arrives in request input and flows
into filesystem paths via find_plugin_dir. Gate it with the same
^[a-zA-Z0-9_-]{1,64}$ allowlist the web UI's pages_v3 uses, at the
single choke point every route resolves through.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): lexical containment check on resolved plugin dirs

CodeQL doesn't recognize the interprocedural allowlist as a
path-injection barrier; add the canonical one — normalize (without
following symlinks, since dev plugins are commonly symlinked into
plugins/) and require the result to stay inside the search dir.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): inline normpath containment barrier before render

CodeQL doesn't credit the sanitization inside find_plugin_dir along
this flow; apply its documented barrier (normpath + startswith against
the allowed roots) inline in _parse_render_request, on the exact path
that reaches the render/load sinks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): derive plugin dir from trusted directory listings

CodeQL's barrier-guard recognition doesn't see a startswith check
inside an any() comprehension, so the normalize-and-prefix approach
still flagged. Break the taint outright instead: after lookup, re-derive
the directory by enumerating the search dirs (iterdir) and matching by
path equality — the Path used for all downstream file access is built
solely from trusted listings, never from request input.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): use os.scandir for path-injection barrier, redact stack traces from render responses

CodeQL doesn't model Path.iterdir() as a taint-clearing enumeration the
way it does os.scandir() -- _trusted_plugin_dir's iterdir-based rebuild
still traced plugin_id through to the manifest.json open(). Switched to
scandir, matching the pattern already verified clean on PR #396.

Also stops surfacing raw exception text (update()/display() failures)
in the JSON render response -- logs full detail server-side via
exc_info instead, returning only the exception class name to the
client. And drops path values from three plugin_loader debug/error
logs that CodeQL flags as clear-text-logging of externally-influenced
data, keeping plugin_id (not flagged) for context.

* fix(dev-server): remove conditional-reassignment ambiguity in plugin_dir resolution

CodeQL's path-injection flow still traced through _parse_render_request
after the scandir fix -- the tainted find_plugin_dir() result and the
scandir-derived _trusted_plugin_dir() result shared the same variable
name (plugin_dir), reassigned only on the truthy branch. That merge
point apparently isn't treated as a barrier by the flow analysis, so it
kept tracing the pre-reassignment value through to the manifest open().

Split into two distinct names -- candidate_dir (tainted, used only to
call _trusted_plugin_dir) and trusted_dir (the only name used for any
downstream file access) -- so there's no reassigned variable for the
flow to walk through.

* fix: remove unused imports flagged by Codacy

Union in adaptive_images.py and field in adaptive_layout.py are both
imported but never used -- the last two Codacy findings on this PR,
matching the same fix already applied on PR #396.

* fix(layout): bound the fit cache; never alias the source image in fits

Two latent issues found in a self-review pass:

- LayoutContext._fit_cache was an unbounded dict (the image cache got an
  LRU cap, the text-fit cache didn't). Cache keys embed the fitted TEXT,
  so a plugin fitting changing strings — a live game clock, a ticker —
  on a 24/7 service grows it forever. Now LRU-bounded at 512 entries via
  the same pattern as the image cache.

- fit_image returned the caller's ORIGINAL image object when the source
  was already RGBA at target size (contain/fill_height, no ink crop).
  ImageFitResult is documented as an independent copy, and LayoutContext
  caches results — an aliased image lets later mutations of the source
  corrupt cached fits (or vice versa). Copy in that branch.

Both covered by new regression tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 10:38:52 -04:00

10 KiB
Raw Permalink Blame History

Adaptive Layout & Font Scaling

src/adaptive_layout.py lets a plugin render legibly on any panel size (64x32, 128x32, 96x48, 128x64, 256x64, ...) without hand-tuned per-display layouts. It is opt-in: nothing changes for plugins that don't use it.

It generalizes three patterns proven in the plugin ecosystem:

Pattern Origin Core API
Geometry scale factor vs. a design size f1-scoreboard ctx.px(base) / ctx.scale
Breakpoint tiers masters-tournament ctx.tier / ctx.by_tier({...})
"Largest crisp font that fits" ladder baseball-scoreboard ctx.fit_text(...) and friends

Quick start

Every BasePlugin has a lazy self.layout (a LayoutContext for the current logical display size, rebuilt automatically if the size changes) and a one-liner self.draw_fit(...):

def display(self, force_clear=False):
    from src.adaptive_layout import LADDER_ARCADE

    b = self.layout.bounds.inset(1)          # Region(0,0,W,H) minus 1px margin
    rows = b.split_v(3, 1, 1, gap=1)          # 3/5 for time, 1/5 each for the rest

    self.draw_fit(self.time_str, rows[0], ladder=LADDER_ARCADE)
    self.draw_fit(self.weekday, rows[1])      # default LADDER_GRID
    self.draw_fit(self.date_str, rows[2])
    self.display_manager.update_display()

On 128x64 the time renders at press_start 24px; on 64x32 it steps down to 8px. The rows partition the height, so bands can never overlap — no more y = height - 7 magic numbers.

Region — rect algebra

Region(x, y, w, h) is a frozen dataclass. All carving clamps to non-negative dimensions, so degenerate panels behave.

  • Carving: inset(dx, dy), top_band(h), bottom_band(h), middle(top_h, bottom_h), left_col(w), right_col(w), split_h(*weights, gap=0), split_v(*weights, gap=0)
  • Placement: align_xy(w, h, align, valign), center_xy(w, h), contains(w, h), .center, .right, .bottom

Scoreboard-style layout:

b = self.layout.bounds
status = b.top_band(self.layout.px(7))
detail = b.bottom_band(self.layout.px(7))
score_area = b.middle(status.h, detail.h)
away_slot, home_slot = b.left_col(b.h), b.right_col(b.h)

Font ladders — discrete, never fractional

Pixel fonts (BDF, PressStart2P) only look right at native/integer sizes, so fonts are never scaled continuously. A FontLadder is an ordered tuple of FontStep(family, size_px) rungs, largest first; fitting walks down until the measured text fits.

  • LADDER_GRID (default): X11 BDFs at native sizes — 10x20 → 9x18 → 9x15 → 8x13 → 7x13 → 6x13 → 6x12 → 6x10 → 6x9 → 5x8 → 5x7 → 4x6 → tom-thumb. Body text, labels, multi-row content.
  • LADDER_ARCADE: PressStart2P at 32/24/16/8 (integer multiples of its 8px grid). Headline text: clocks, scores.

Custom ladders are just tuples — e.g. to add your plugin's registered font on top: (FontStep("myplugin::digits", 16),) + LADDER_GRID.

LayoutContext

Built per (width, height); exposes facts and fit queries:

  • bounds, width, height, aspect
  • tier by height (xs≤16, sm≤32, md≤48, lg≤64, xl) and width_tier (narrow≤64, normal≤128, wide≤256, ultrawide)
  • is_wide_short — aspect ≥ 2.5 and height ≤ 32 (the classic 128x32 shape)
  • scalemin(w/design_w, h/design_h) vs. your manifest's display.design_size (default 128x32). Geometry only — gaps, icon and logo sizes via px(base, minimum, maximum); fonts use ladders.
  • by_tier({"sm": 10, "lg": 18}) — value for the nearest defined tier at-or-below the panel's tier.
  • fit_text(text, box, ladder, ellipsis=True)FitResult — largest rung that fits; ellipsizes as a last resort. Cached per (text, box, ladder).
  • fit_text_proportional(text, box, base_size_px, ladder, ellipsis=True, scale=None) — rung closest to (not exceeding) base_size_px * scale, still capped to what fits the box. Use this instead of fit_text when several independently-fitted elements need to stay visually harmonious as the panel grows — fit_text maximizes each one within its own region, which can make one element (e.g. a score with a generous box) balloon out of proportion to a neighbor that scales by geometry (e.g. logos sized via px()), even though each individual pick is "correct" in isolation. base_size_px is normally the element's existing classic/ fixed font size. scale defaults to self.scale (the conservative min-of-both-axes factor px() uses); pass an axis-specific value when the surrounding composition already scales that way — e.g. a scoreboard whose logo slots track height alone (min(height, width // 2)) should size its text by height / design_height too, or the text reads as under-scaled next to bigger logos on a panel that only grew taller.
  • fit_lines(lines, box, ladder, spacing) — every line fits the width and the stack fits the height (measures the actual strings).
  • font_for_rows(rows, box_h, ladder) — largest rung whose line height fits rows rows.

FitResult carries the ready-to-use font (drops straight into display_manager.draw_text(font=...)), the possibly-ellipsized text, ink width/height, baseline, y_offset, line_height, and fits.

Adaptive images

src/adaptive_images.py is the image counterpart to fit_text, exposed as self.layout.fit_image(...) (cached per panel size) and the one-liner self.draw_image(...):

# Team logo: trim its transparent padding, fill the slot height (the
# football/hockey pattern), cached across frames by a stable key
self.draw_image(logo, regs.away_slot, mode="fill_height",
                crop_to_ink=True, cache_key=f"logo:{abbr}")

# Album art: cover-crop a square, faces kept by the top anchor
self.draw_image(art, row.art, mode="cover", anchor="top")

# Pixel flags / sprite icons: NEAREST keeps hard edges
from src.adaptive_images import RESAMPLE_NEAREST
self.draw_image(flag, box, resample=RESAMPLE_NEAREST)

Modes: contain (letterbox, default), cover (crop-to-fill), fill_height (logo-style), stretch. Unlike PIL's thumbnail() (downscale-only — why imagery stays tiny on big panels) fitting upscales by default; pass upscale=False for the legacy behavior. Results are cached per (image, box size, options) with a bounded LRU — always pass a stable cache_key (e.g. "logo:KC") for images you reload. The module also exports the Pillow-compat RESAMPLE_LANCZOS/RESAMPLE_NEAREST constants so plugins can drop their local shims.

Composite layouts

Pre-carved Region arrangements for the layouts plugins keep rebuilding:

from src.adaptive_layout import scoreboard_regions, media_row

regs = scoreboard_regions(self.layout.bounds, ctx=self.layout)
# regs.away_slot / home_slot   — logo slots (logo_slot = min(H, W // 2),
#                                 capped so a center reserve always exists —
#                                 see below)
# regs.status_band             — top band (replaces the magic y = 1)
# regs.score_area              — center gap, plus a controlled bleed into
#                                 each logo slot (replaces y = H//2 - 3)
# regs.detail_band             — bottom band (replaces y = H - 7)
# regs.bottom_left / bottom_right — record/timeout corners

row = media_row(self.layout.bounds, ctx=self.layout)   # art left, text right

Both work on the full panel or on a scroll-mode card Region. They return Regions and never draw — compose them with draw_fit/draw_image.

scoreboard_regions's center reserve. The raw logo_slot = min(H, W//2) formula has a blind spot: at exactly 2:1 aspect ratio (width = 2×height — two, four, or more square modules stacked into a taller panel, e.g. 96x48, 128x64, 256x128) the two logo slots mathematically claim the entire width, leaving zero pixels for a center column no matter how big the panel gets. Wide panels (the 128x32 design baseline, 192x48, 256x32) never hit this, since height is already the tighter constraint there. Two parameters fix it without any plugin-side code: min_center_fraction/min_center_design_px guarantee a real minimum center reserve at any aspect ratio, and score_bleed_fraction lets the score's fit box extend a controlled amount into each logo slot — the same way a real broadcast scoreboard's numbers cross slightly into the team marks flanking them — so a short score string never has to truncate even on the tightest aspect ratios. All three have sane defaults; override them per call if a plugin's card proportions genuinely differ.

Preserving user customization

Adaptive layout supplies defaults; explicit user configuration wins:

  • User-set fonts win. If the plugin's config has an explicit font/font_size for an element, load it as before and skip the ladder — fit only when the user hasn't overridden (see the football-scoreboard _resolve_element_fit pattern).
  • Offsets apply on top. customization.layout.<element>.{x_offset,y_offset} style knobs translate the computed region as a final step: region.offset(user_dx, user_dy). draw_image(..., offset=(dx, dy)) does the same for images.
  • Colors pass through. draw_fit/draw_fitted_text take explicit color= params; adaptive mode never repaints semantic or user-chosen colors.

Manifest declaration

Declare the size your layout was authored against so ctx.scale means something:

"display": { "design_size": { "width": 128, "height": 32 } }

Also available under requires.display_size: min_width, min_height, max_width, max_height.

Performance notes (Pi)

Fit queries are cached, so cost is O(unique strings). For per-second text (clocks, live scores), fit on a shape placeholder and reuse the font:

fit = self.layout.fit_text("00:00", box, ladder=LADDER_ARCADE)  # cached once
self.display_manager.draw_text(current_time, font=fit.font, ...)

Testing across sizes

The harness already renders every plugin at a spread of sizes (now including 96x48):

python scripts/check_plugin.py <plugin-dir> --sizes 64x32,128x32,96x48,128x64,256x64
python scripts/render_plugin.py <plugin-dir> --width 96 --height 48

BoundsCheckingDisplayManager flags right/bottom overflow and now records mediated draw calls with negative coordinates in negative_coordinate_calls (raw-PIL draws remain uncovered).

Reference migration: the text-display plugin's font_mode: "auto".