Files
LEDMatrix/test/test_api_v3_partial_main_save.py
ChuckandClaude Opus 5.5 7f06cc9c3b feat(vegas): keep live games in the ticker by default (#699)
* perf(timing): say which render-thread work a late frame followed

The soak already says how often a moving frame reached the panel late, but
not what the render thread was doing just before it. Vegas does two kinds of
work there between frames -- building its strip (compose, extend) and, with
live elements, patching changed pixels into it -- and deciding whether either
is affordable needs their own numbers.

- FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame.
  Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind;
  aggregate() still takes frames without ops. The file schema is unchanged.
- Vegas tags compose and every strip extension (with the bytes it copied).
- frame_soak prints an "after work" table: frames, late %, freezes and MB
  moved per kind, only when something tagged its work.
- render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes /
  --patch-every / --patch-where (in-place column writes, as a live element
  update does) and --extend-every-screens / --extend-width (append + trim on
  a fixed cadence that holds the strip's width).

No runtime behaviour changes: this is the measurement gate for live Vegas
elements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): note the frame-op attribution and bench modes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* perf(scroll): build the strip's PIL image only when something reads it

Every Vegas strip extension rebuilt ScrollHelper.cached_image from
cached_array in full, twice (append, then trim), on the render thread:
Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a
Pi 4 (measured on ledpi), about two thirds of an extension's render-thread
cost. Nothing on the frame path reads the image's pixels; every frame is cut
from the array.

cached_image is now a property. append_content and drop_scrolled_prefix
defer it; the first read builds it from the array it started with and keeps
it only if the strip has not changed meanwhile, so a sync push racing an
extension cannot leave a stale image cached. Assigning cached_image stores
exactly what was assigned, as before. has_strip() says whether there is a
strip without building its image; the helper's frame path, Vegas and the
adapter's scroll-cache invalidation use it. The strip is also no longer held
in memory twice.

In Vegas the image is now built only by a multi-display sync push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(vegas): live elements -- a plugin API for content that changes while it scrolls

Vegas bakes each plugin's pictures into one strip, so a card already on its
way across the panel keeps what it showed when it was drawn. This adds the
API and bookkeeping for content that can be updated in place; the worker
that redraws and swaps it follows separately. No shipped plugin implements
the hook yet, so nothing changes for users.

Plugin API (core 3.8.0), all no-ops by default:
- BasePlugin.get_vegas_elements() -> [VegasElement(key, image, version,
  live, refresh_hz)]: named, fixed-width pieces of Vegas content.
- BasePlugin.redraw_vegas_element(key, width, height, at): a lock-free
  redraw for content that changes with time.
- BasePlugin.notify_vegas_data_changed(): data that lands outside update().
- src/plugin_system/vegas_elements.py (VegasElement, re-exported from
  base_plugin).

Core:
- PluginAdapter asks a plugin that implements the hook for elements on the
  background fetch only (under its lock, on its own canvas); every other
  path keeps get_vegas_content(). Live elements are pinned (padded with
  content_padding, never trimmed), tagged with their key, digest and data
  epoch in Image.info so the existing cache and group plumbing carry them
  unchanged, and untagged if a width budget crops them.
- RenderPipeline records where each live element lands (ElementRecord), in
  absolute strip columns a trim does not move; the block-start arithmetic
  is shared with the STATIC markers.
- PluginManager update listeners (add/remove_update_listener,
  notify_data_changed): told the moment update() completes, not at the
  next ~4s Vegas poll. The coordinator uses one to move each plugin's data
  epoch on.
- vegas_scroll.live_refresh (kill switch), live_max_hz, live_min_interval,
  live_lead_screens; per-plugin core-owned vegas_live. Live elements are
  off under multi-display sync, in swap mode and with offscreen_prefetch off.
- scripts/check_plugin.py checks the element contract
  (src/plugin_system/testing/vegas.py); test/fixtures/plugins/vegas-live-stub
  is a working example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(vegas): live elements update in place while they scroll

One background worker (src/vegas_mode/live_worker.py) redraws a plugin's
live elements when its data epoch moves on (update listener) or on their
refresh_hz, nearest the screen first, and hands changed pixels lock-free to
the render thread, which copies them into the strip between frames
(RenderPipeline.apply_live_patches, ScrollHelper.patch_columns): at most
four patches or two screens of bytes a frame, no drawing or locks there.
The worker takes over group prefetch once a live element is placed, runs
inside the render gate, and is supervised. Update tick 1s while live
elements exist. Web UI switch for live_refresh. OFFSCREEN_RENDERING.md
describes what was built and why SegmentStrip was not needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(sports): live Vegas cards for the scoreboards (shared layer)

One live element per game, drawn only when what the card shows changes, so
a score changes on a card already crossing the panel. The shared part, so
each scoreboard adopts it in a few lines:

- src/common/sports_vegas.py: game_key, game_fingerprint (the whole game
  dict, frozen: no drawn field can be missed), dedupe_games, VegasCardCache,
  StickyOdds (odds a live poll left out stay drawn), finished_games /
  with_finished_games (a game that just went final keeps its card, after its
  league's live games; one a heuristic only judged over keeps its live
  state, so a tied end of regulation never shows FINAL early).
- SportsScrollDisplay.make_vegas_renderer() is the override point;
  build_vegas_elements() and SportsScrollDisplayManager
  .get_vegas_elements_for() do the rest. A card's version includes its
  teams' ranks, which the renderer draws from the rankings cache.
- SportsLiveSharedMixin._record_finished_game() / finished_games_snapshot():
  held for FINISHED_GAME_TTL after it leaves the live list.

A sport that does not implement make_vegas_renderer keeps its ordinary Vegas
content, so no scoreboard changes until it opts in.

scripts/render_plugin.py --vegas renders a plugin's Vegas block as the
ticker lays it out, and --timeline stacks it at successive moments as
the ticker would update it in place; the join is now
render_pipeline.join_plugin_rows().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(vegas): keep live games in the ticker by default

display.vegas_scroll.live_in_ticker now defaults to true: through a live
game the marquee keeps running and the live scoreboard takes extra turns in
it -- its cards updating in place while they scroll -- instead of the ticker
giving way to the full-screen scoreboard.

The new default would reach nobody on its own: every existing config holds
an explicit false copied from the template (there was no control for it),
and the template merge only adds missing keys. ConfigManager therefore turns
a stored false on once, with a backup, and records live_in_ticker_migrated
so a false chosen afterwards stays. The marker is never in the template.

A "Keep live games in the ticker" checkbox under Vegas mode sets it. Tests
that pin the full-screen takeover now say live_in_ticker=false.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(sports): a default _determine_game_type on SportsScrollDisplay

render_vegas_card looked the method up with getattr and a None default, which
static analysis (Codacy) reports as calling something that may not be
callable. The base class now has the default -- the card type from the game's
state -- and the plugins that define their own override it as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: review follow-ups on the shared live-card layer

- The reused Vegas renderer always gets the current rankings, empty
  included, so ranks cleared since are not kept drawn.
- render_plugin.py: --timeline refuses --no-live (a timeline shows live
  elements changing), --timeline/--no-live need --vegas, and the Vegas
  paths create the output's directory like the display path does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 08:27:16 -04:00

331 lines
17 KiB
Python

"""POST /config/main: a partial JSON body changes only what it sends.
The settings forms post every field, and a browser leaves an unchecked box out,
so for a form a missing checkbox means False. JSON API clients send only what
they change. Treating their missing keys as unchecked meant:
- the MQTT bridge's Home Assistant brightness slider (``{"brightness": N}``)
turned off disable_hardware_pulsing, inverse_colors, show_refresh_rate and
use_short_date_format on every change;
- the documented timezone/location update turned off web_display_autostart and
weekly automatic updates.
The v3 forms post JSON as well (htmx json-enc), so they mark themselves with a
hidden ``__form_section`` input; these tests pin both halves of that contract.
Also here: the Vegas cycle-time fields no longer land in display_durations
(and a blank one no longer 400s the Display save), the Raw JSON editor starts
auto-update setup like the General form does, and both schedule POSTs accept
the per-day shape their GETs return.
"""
import copy
import json
import re
from pathlib import Path
import pytest
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401
REPO = Path(__file__).resolve().parent.parent
PARTIALS = REPO / 'web_interface' / 'templates' / 'v3' / 'partials'
STORED = {
'web_display_autostart': True,
'auto_update': {'enabled': True},
'timezone': 'America/Chicago',
'plugin_system': {'auto_discover': True, 'auto_load_enabled': True,
'development_mode': True},
'display': {
'hardware': {'rows': 32, 'cols': 64, 'chain_length': 2, 'brightness': 90,
'disable_hardware_pulsing': True, 'inverse_colors': True,
'show_refresh_rate': True},
'runtime': {'gpio_slowdown': 4},
'use_short_date_format': True,
'double_sided': {'enabled': True, 'copies': 2, 'axis': 'horizontal'},
'vegas_scroll': {'enabled': True, 'auto_trim': True,
'dynamic_duration_enabled': True,
'continuous_scroll': True, 'smooth_scroll': True},
},
}
@pytest.fixture
def saved(api_v3_module, monkeypatch):
captured = {}
api_v3_module.api_v3.config_manager.load_config.side_effect = \
lambda *a, **k: copy.deepcopy(STORED)
def fake_save(_manager, config, **_kwargs):
captured['config'] = config
return True, ''
monkeypatch.setattr(api_v3_module, '_save_config_atomic', fake_save)
from web_interface import auto_update
monkeypatch.setattr(auto_update, 'start_setup_if_needed', lambda *a, **k: None)
return captured
def _post_json(client, body):
return client.post('/api/v3/config/main', data=json.dumps(body),
content_type='application/json')
class TestJsonPartialSaves:
def test_mqtt_bridge_brightness_changes_only_brightness(self, api_v3_client, saved):
# integrations/mqtt_bridge/ledmatrix_mqtt_bridge.py set_brightness
resp = _post_json(api_v3_client, {'brightness': 40})
assert resp.status_code == 200, resp.get_json()
display = saved['config']['display']
assert display['hardware']['brightness'] == 40
assert display['hardware']['disable_hardware_pulsing'] is True
assert display['hardware']['inverse_colors'] is True
assert display['hardware']['show_refresh_rate'] is True
assert display['use_short_date_format'] is True
def test_timezone_only_keeps_autostart_and_auto_update(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'timezone': 'UTC'})
assert resp.status_code == 200, resp.get_json()
config = saved['config']
assert config['timezone'] == 'UTC'
assert config['web_display_autostart'] is True
assert config['auto_update'] == {'enabled': True}
def test_location_only_keeps_autostart_and_auto_update(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'city': 'Paris', 'country': 'FR'})
assert resp.status_code == 200, resp.get_json()
config = saved['config']
assert config['location'] == {'city': 'Paris', 'country': 'FR'}
assert config['web_display_autostart'] is True
assert config['auto_update'] == {'enabled': True}
@pytest.mark.parametrize('key', ['auto_discover', 'auto_load_enabled', 'development_mode'])
def test_a_legacy_plugin_system_toggle_is_not_a_general_save(self, api_v3_client, saved, key):
# These left the General form; a client still sending one must not
# have the general-settings checkboxes treated as unchecked.
resp = api_v3_client.post('/api/v3/config/main', data={key: 'on'},
content_type='application/x-www-form-urlencoded')
assert resp.status_code == 200, resp.get_json()
assert saved['config']['web_display_autostart'] is True
assert saved['config']['auto_update'] == {'enabled': True}
def test_vegas_speed_only_keeps_vegas_toggles(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'vegas_scroll_speed': 80})
assert resp.status_code == 200, resp.get_json()
vegas = saved['config']['display']['vegas_scroll']
assert vegas['scroll_speed'] == 80
for key in ('enabled', 'auto_trim', 'dynamic_duration_enabled',
'continuous_scroll', 'smooth_scroll'):
assert vegas[key] is True, key
def test_vegas_speed_only_keeps_live_updates_as_stored(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'vegas_scroll_speed': 70})
assert resp.status_code == 200, resp.get_json()
assert 'live_refresh' not in saved['config']['display']['vegas_scroll']
def test_live_updates_can_be_switched_off_and_on(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'vegas_live_refresh': False})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['vegas_scroll']['live_refresh'] is False
resp = _post_json(api_v3_client, {'__form_section': 'display',
'vegas_scroll_speed': '50',
'vegas_live_refresh': 'on'})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['vegas_scroll']['live_refresh'] is True
def test_live_games_can_be_kept_in_the_ticker_or_not(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'vegas_live_in_ticker': False})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['vegas_scroll']['live_in_ticker'] is False
# An unticked box is absent from a submitted form: that is false too.
resp = _post_json(api_v3_client, {'__form_section': 'display',
'vegas_scroll_speed': '50',
'vegas_live_in_ticker': 'on'})
assert saved['config']['display']['vegas_scroll']['live_in_ticker'] is True
resp = _post_json(api_v3_client, {'__form_section': 'display',
'vegas_scroll_speed': '50'})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['vegas_scroll']['live_in_ticker'] is False
def test_double_sided_axis_only_keeps_enabled(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'double_sided_axis': 'horizontal'})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['double_sided']['enabled'] is True
def test_json_can_still_turn_a_checkbox_off(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'inverse_colors': False, 'auto_update_enabled': False})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['hardware']['inverse_colors'] is False
assert saved['config']['display']['hardware']['disable_hardware_pulsing'] is True
assert saved['config']['auto_update'] == {'enabled': False}
def test_json_with_charset_is_still_json(self, api_v3_client, saved):
resp = api_v3_client.post('/api/v3/config/main', data=json.dumps({'brightness': 41}),
content_type='application/json; charset=utf-8')
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['hardware']['brightness'] == 41
assert saved['config']['display']['hardware']['inverse_colors'] is True
def test_a_non_object_body_is_refused(self, api_v3_client, saved):
assert _post_json(api_v3_client, [1, 2]).status_code == 400
class TestFormSavesStillUncheck:
"""Unchecking a box in the UI must still save false."""
def test_marked_json_form_post_unchecks_missing_display_boxes(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'__form_section': 'display', 'brightness': '40',
'vegas_scroll_speed': '50'})
assert resp.status_code == 200, resp.get_json()
display = saved['config']['display']
for key in ('disable_hardware_pulsing', 'inverse_colors', 'show_refresh_rate'):
assert display['hardware'][key] is False, key
assert display['use_short_date_format'] is False
assert display['vegas_scroll']['enabled'] is False
assert '__form_section' not in saved['config']
def test_marked_json_form_post_keeps_checked_boxes(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'__form_section': 'display', 'brightness': '40',
'inverse_colors': 'on'})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['hardware']['inverse_colors'] is True
assert saved['config']['display']['hardware']['show_refresh_rate'] is False
def test_marked_general_form_unchecks_autostart_and_auto_update(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'__form_section': 'general', 'timezone': 'UTC'})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['web_display_autostart'] is False
assert saved['config']['auto_update'] == {'enabled': False}
def test_form_encoded_post_unchecks_missing_boxes(self, api_v3_client, saved):
resp = api_v3_client.post('/api/v3/config/main', data={'brightness': '40'},
content_type='application/x-www-form-urlencoded')
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['hardware']['inverse_colors'] is False
@pytest.mark.parametrize('partial', ['general.html', 'display.html', 'durations.html'])
def test_every_config_main_form_carries_the_marker(self, partial):
html = (PARTIALS / partial).read_text(encoding='utf-8')
form = re.search(r'<form hx-post="/api/v3/config/main".*?</form>', html, re.S)
assert form, f'{partial} no longer posts to /config/main'
assert re.search(r'<input type="hidden" name="__form_section" value="\w+">',
form.group(0)), f'{partial} form lost its __form_section marker'
class TestVegasCycleDurations:
def test_cycle_durations_are_not_display_durations(self, api_v3_client, saved):
resp = api_v3_client.post('/api/v3/config/main', data={
'vegas_scroll_enabled': 'on', 'vegas_min_cycle_duration': '90',
'vegas_max_cycle_duration': '300'})
assert resp.status_code == 200, resp.get_json()
display = saved['config']['display']
assert display['vegas_scroll']['min_cycle_duration'] == 90
assert display['vegas_scroll']['max_cycle_duration'] == 300
durations = display.get('display_durations', {})
assert 'vegas_min_cycle_duration' not in durations
assert 'vegas_max_cycle_duration' not in durations
assert 'vegas_min_cycle_duration' not in saved['config']
def test_blank_cycle_duration_does_not_reject_the_save(self, api_v3_client, saved):
resp = api_v3_client.post('/api/v3/config/main', data={
'vegas_scroll_enabled': 'on', 'vegas_scroll_speed': '60',
'vegas_min_cycle_duration': ''})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['vegas_scroll']['scroll_speed'] == 60
def test_real_display_durations_still_save(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'clock_duration': '45'})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['display_durations']['clock_duration'] == 45
def test_per_mode_duration_saves_and_blank_clears_it(self, api_v3_client, saved, api_v3_module):
# The Rotation page leaves a mode blank to mean "the plugin's own
# duration"; a saved value overrides the plugin, so blank must remove
# it rather than 400 or pin a number.
stored = copy.deepcopy(STORED)
stored['display']['display_durations'] = {'weather_current': 40, 'clock': 20}
api_v3_module.api_v3.config_manager.load_config.side_effect = lambda *a, **k: copy.deepcopy(stored)
resp = _post_json(api_v3_client, {'__form_section': 'durations',
'duration__clock': '45',
'duration__weather_current': ''})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['display_durations'] == {'clock': 45}
class TestRawSaveStartsAutoUpdateSetup:
@pytest.fixture
def raw_env(self, api_v3_module, monkeypatch):
cm = api_v3_module.api_v3.config_manager
cm.get_raw_file_content.return_value = {'timezone': 'UTC', 'auto_update': {'enabled': False}}
calls = []
from web_interface import auto_update
monkeypatch.setattr(auto_update, 'start_setup_if_needed',
lambda was, config: calls.append((was, config)) or 'Setup note.')
return cm, calls
def test_enabling_from_raw_json_calls_setup(self, api_v3_client, raw_env):
cm, calls = raw_env
body = {'timezone': 'UTC', 'auto_update': {'enabled': True}}
resp = api_v3_client.post('/api/v3/config/raw/main', json=body)
assert resp.status_code == 200, resp.get_json()
cm.save_raw_file_content.assert_called_once_with('main', body)
assert calls == [(False, body)]
assert 'Setup note.' in resp.get_json()['message']
def test_previously_enabled_is_passed_through(self, api_v3_client, raw_env):
cm, calls = raw_env
cm.get_raw_file_content.return_value = {'auto_update': {'enabled': True}}
body = {'auto_update': {'enabled': True}}
assert api_v3_client.post('/api/v3/config/raw/main', json=body).status_code == 200
assert calls == [(True, body)]
class TestSchedulesAcceptTheirGetShape:
PER_DAY = {
'enabled': True, 'mode': 'per-day', 'dim_brightness': 20,
'days': {
'monday': {'enabled': True, 'start_time': '21:15', 'end_time': '06:45'},
'tuesday': {'enabled': False},
'wednesday': {'enabled': True, 'start_time': '22:00', 'end_time': '05:30'},
'thursday': {'enabled': True, 'start_time': '20:00', 'end_time': '07:00'},
'friday': {'enabled': True, 'start_time': '23:00', 'end_time': '08:00'},
'saturday': {'enabled': True, 'start_time': '23:30', 'end_time': '09:00'},
'sunday': {'enabled': True, 'start_time': '21:00', 'end_time': '07:00'},
},
}
@pytest.fixture
def store(self, api_v3_module, monkeypatch):
state = {'config': {}}
api_v3_module.api_v3.config_manager.load_config.side_effect = \
lambda *a, **k: copy.deepcopy(state['config'])
def fake_save(_manager, config, **_kwargs):
state['config'] = copy.deepcopy(config)
return True, ''
monkeypatch.setattr(api_v3_module, '_save_config_atomic', fake_save)
return state
@pytest.mark.parametrize('route,section,extra', [
('/api/v3/config/dim-schedule', 'dim_schedule', {'dim_brightness': 20}),
('/api/v3/config/schedule', 'schedule', {}),
])
def test_get_output_posts_back_unchanged(self, api_v3_client, store, route, section, extra):
body = {k: v for k, v in self.PER_DAY.items() if k != 'dim_brightness'}
body.update(extra)
store['config'] = {section: copy.deepcopy(body)}
read = api_v3_client.get(route).get_json()['data']
resp = api_v3_client.post(route, json=read)
assert resp.status_code == 200, resp.get_json()
assert store['config'][section]['days'] == body['days']
def test_flat_form_keys_still_work(self, api_v3_client, store):
body = {'enabled': True, 'mode': 'per-day', 'dim_brightness': 25,
'monday_enabled': 'on', 'monday_start': '19:00', 'monday_end': '06:00'}
resp = api_v3_client.post('/api/v3/config/dim-schedule', json=body)
assert resp.status_code == 200, resp.get_json()
assert store['config']['dim_schedule']['days']['monday'] == {
'enabled': True, 'start_time': '19:00', 'end_time': '06:00'}