fix(web): stop checkbox groups posting back options they cannot show (#465)

The enum that lets a checkbox group draw its options is also what validates
the saved value. When an option goes away -- a league retires a team code, a
schema drops a choice -- a config still holding the old value has no checkbox
to render for it, but the value stayed in the hidden _data input anyway:
that input is seeded from the stored array and only rebuilt by
updateCheckboxGroupData() on change.

So the stale value was posted back on every save the user did not happen to
touch that widget for. The schema rejected it and the save endpoint returned
400 CONFIG_VALIDATION_FAILED, which blocks editing *any* field on that
plugin until the user works out which invisible entry is at fault -- with
nothing on screen naming it, because the offending value is precisely the one
with no checkbox.

Runtime was never affected: load_plugin() treats schema violations as
warn/degrade, and a retired code already matched nothing. Only the web UI
blocked.

Values not in the enum are now dropped before the hidden input is seeded, and
listed above the group so the selection is not lost silently. Only when the
widget has options -- an empty enum means there is nothing to check against,
and filtering on it would wipe the field.

This is not hypothetical. ledmatrix-plugins #212 ("correct team abbreviations
so config save no longer 400s") and #234 (removed the retired NHL code UTA
from a picker across four plugins) are both this failure mode, fixed one
league at a time. Nine shipped plugins use checkbox-group today; all of them
get the fix.

Tested by rendering the checkbox-group block lifted out of the shipped
template, following test_enum_option_labels.py, so the tests exercise the
production expression rather than a copy. Mutation-checked: removing the
filter fails 2 tests, filtering unconditionally fails the empty-enum test,
and dropping the notice fails the one asserting the value is named.

test/test_checkbox_group_stale_values.py

@@ -0,0 +1,113 @@
+"""A checkbox group must not post back options it cannot show.
+
+The enum that lets the widget draw checkboxes is also what validates the
+saved value. When a league retires a team code -- OAK for the Athletics, ARI
+for the Coyotes -- or a schema drops an option, a config that still holds the
+old value has nothing to render for it. The value stayed in the hidden
+``_data`` input regardless, because that input is seeded from the stored array
+and only rebuilt by ``updateCheckboxGroupData()`` on change. Editing any other
+field on that plugin therefore posted the stale value back, the schema
+rejected it, and the save endpoint returned 400
+``CONFIG_VALIDATION_FAILED`` -- so the whole plugin became uneditable until
+the user worked out which invisible entry was at fault.
+
+Runtime was never affected: plugin loading treats schema violations as
+warn/degrade, and the stale code already matched no team. Only the web UI
+blocked.
+
+These tests render the checkbox-group block lifted *out of the shipped
+template*, following test_enum_option_labels.py, so they exercise the
+production expression rather than a copy that could drift from it.
+"""
+import json
+import re
+from pathlib import Path
+
+from jinja2 import DictLoader, Environment
+
+PROJECT_ROOT = Path(__file__).resolve().parent.parent
+CONFIG_FORM = (PROJECT_ROOT / 'web_interface' / 'templates' / 'v3' / 'partials'
+               / 'plugin_config.html')
+
+# The checkbox-group branch: from its `{% elif %}` guard through the sentinel
+# hidden input that closes it. Anchored on the guard so the match cannot run on
+# into a neighbouring widget branch.
+BLOCK_RE = re.compile(
+    r"\{%\s*elif x_widget == 'checkbox-group'\s*%\}(.*?)"
+    r"<input type=\"hidden\" name=\"\{\{ full_key \}\}\[\]\" value=\"\">",
+    re.S,
+)
+
+
+def _shipped_block() -> str:
+    """Return the live checkbox-group block lifted from plugin_config.html."""
+    source = CONFIG_FORM.read_text(encoding='utf-8')
+    match = BLOCK_RE.search(source)
+    assert match, (
+        'could not find the checkbox-group block in plugin_config.html — the '
+        'template changed shape and this guard needs updating'
+    )
+    block = match.group(1)
+    assert 'data-option-value' in block, 'extracted the wrong branch'
+    assert '{% elif' not in block, 'extraction ran past the checkbox-group branch'
+    return block
+
+
+def _render(prop: dict, value=None) -> str:
+    env = Environment(loader=DictLoader({'f': _shipped_block()}), autoescape=True)
+    return env.get_template('f').render(
+        prop=prop, value=value, field_id='fid', full_key='k'
+    )
+
+
+def _submitted(html: str) -> list:
+    """The array the form will actually post: the hidden _data input."""
+    match = re.search(r'id="fid_data"[^>]*\svalue=\'([^\']*)\'', html)
+    assert match, f'hidden _data input not found in:\n{html}'
+    return json.loads(match.group(1).replace('&#39;', "'"))
+
+
+def _checked(html: str) -> list:
+    return re.findall(r'data-option-value="([^"]+)"[^>]*checked', html)
+
+
+MLB = {'type': 'array', 'items': {'type': 'string', 'enum': ['NYY', 'BOS', 'ATH']},
+       'x-widget': 'checkbox-group'}
+
+
+def test_a_retired_code_is_not_posted_back() -> None:
+    """The regression: OAK became ATH, and OAK used to ride along on save."""
+    html = _render(MLB, ['NYY', 'OAK'])
+    assert _submitted(html) == ['NYY'], 'stale value would still be submitted'
+
+
+def test_the_dropped_value_is_named_rather_than_vanishing() -> None:
+    html = _render(MLB, ['NYY', 'OAK'])
+    assert 'OAK' in html
+    assert 'data-stale-options' in html
+
+
+def test_valid_values_are_untouched_and_still_checked() -> None:
+    html = _render(MLB, ['NYY', 'ATH'])
+    assert _submitted(html) == ['NYY', 'ATH']
+    assert sorted(_checked(html)) == ['ATH', 'NYY']
+    assert 'data-stale-options' not in html
+
+
+def test_an_all_stale_selection_clears_rather_than_blocking() -> None:
+    html = _render(MLB, ['OAK', 'SD'])
+    assert _submitted(html) == []
+
+
+def test_an_empty_enum_leaves_the_value_alone() -> None:
+    """No options means nothing to validate against — filtering would wipe it."""
+    prop = {'type': 'array', 'items': {'type': 'string'}, 'x-widget': 'checkbox-group'}
+    html = _render(prop, ['ANYTHING', 'GOES'])
+    assert _submitted(html) == ['ANYTHING', 'GOES']
+
+
+def test_unset_value_falls_back_to_the_default() -> None:
+    prop = dict(MLB, default=['BOS'])
+    html = _render(prop, None)
+    assert _submitted(html) == ['BOS']
+    assert _checked(html) == ['BOS']

test/test_onboarding_checklist.py

@@ -0,0 +1,241 @@
+"""
+Getting Started checklist: what the server decides, and what it must not.
+
+The timezone step used to tick server-side when the saved timezone differed
+from the shipped default, OR-ed with the saved city. That made the step
+unsatisfiable for anyone genuinely in the default zone (the card nagged
+forever), and let a saved city tick it off while the timezone was still wrong.
+The step is now verified in the browser against its own zone, so the server's
+only job is to hand over the configured value and stay out of the decision.
+
+These tests pin that contract: the panel-size step still reflects config, the
+timezone step never pre-ticks, it carries the configured zone, and the city
+has no influence on it.
+"""
+
+import copy
+import re
+import sys
+from pathlib import Path
+from unittest.mock import MagicMock
+
+import pytest
+from flask import Flask
+
+PROJECT_ROOT = Path(__file__).parent.parent
+sys.path.insert(0, str(PROJECT_ROOT))
+
+BASE_CONFIG = {
+    "timezone": "America/New_York",
+    "location": {"city": "Tampa", "state": "Florida", "country": "US"},
+    "display": {
+        "hardware": {"rows": 32, "cols": 64, "chain_length": 2, "parallel": 1},
+        "runtime": {},
+        "double_sided": {"enabled": False},
+        "vegas_scroll": {"plugin_order": [], "excluded_plugins": []},
+        "plugin_rotation_order": [],
+    },
+    "plugin_system": {},
+    "schedule": {},
+    "dim_schedule": {},
+    "sync": {},
+}
+
+
+def render(config):
+    """Render the overview partial against one config, as app.py would."""
+    base = PROJECT_ROOT / "web_interface"
+    app = Flask(
+        __name__,
+        template_folder=str(base / "templates"),
+        static_folder=str(base / "static"),
+    )
+    app.config["TESTING"] = True
+
+    from web_interface.blueprints import pages_v3 as pv
+
+    # pages_v3 is a module-level singleton shared across the test process;
+    # restore whatever the previous test left on it.
+    original_cm = getattr(pv.pages_v3, "config_manager", None)
+    original_pm = getattr(pv.pages_v3, "plugin_manager", None)
+
+    mock_cm = MagicMock()
+    mock_cm.load_config.return_value = config
+    mock_cm.get_raw_file_content.return_value = config
+    pv.pages_v3.config_manager = mock_cm
+
+    mock_pm = MagicMock()
+    mock_pm.plugins = {}
+    mock_pm.get_all_plugin_info.return_value = []
+    mock_pm.get_plugin_display_modes.side_effect = lambda pid: []
+    pv.pages_v3.plugin_manager = mock_pm
+
+    app.register_blueprint(pv.pages_v3, url_prefix="")
+    try:
+        resp = app.test_client().get("/partials/overview")
+        assert resp.status_code == 200, resp.status_code
+        return resp.get_data(as_text=True)
+    finally:
+        pv.pages_v3.config_manager = original_cm
+        pv.pages_v3.plugin_manager = original_pm
+
+
+def timezone_step(body):
+    """The checklist <button> for the timezone step."""
+    match = re.search(r"<button[^>]*data-check=\"timezone\"[^>]*>", body)
+    assert match, "timezone step not found in the rendered checklist"
+    return match.group(0)
+
+
+def config_with(**overrides):
+    config = copy.deepcopy(BASE_CONFIG)
+    for key, value in overrides.items():
+        config[key] = value
+    return config
+
+
+@pytest.mark.parametrize(
+    "timezone",
+    ["America/New_York", "America/Los_Angeles", "Europe/Madrid", "Asia/Kolkata"],
+)
+def test_timezone_step_never_pre_ticks_server_side(timezone):
+    """The browser owns this decision; the server must not pre-empt it.
+
+    The default zone is in the list deliberately: that is the case the old
+    default-comparison could never tick.
+    """
+    step = timezone_step(render(config_with(timezone=timezone)))
+    assert 'data-done="0"' in step, step
+
+
+@pytest.mark.parametrize(
+    "timezone",
+    ["America/New_York", "Europe/Madrid", "Pacific/Auckland"],
+)
+def test_timezone_step_carries_the_configured_zone(timezone):
+    """JS compares data-tz against the browser, so it has to be the real value."""
+    assert f'data-tz="{timezone}"' in timezone_step(render(config_with(timezone=timezone)))
+
+
+def test_city_does_not_influence_the_timezone_step():
+    """The coupling this change removes: city said nothing about the timezone,
+    and OR-ing it let a saved city tick the step off with the zone still wrong.
+
+    timezone_step() returns the opening tag only, so this compares the state
+    the step is in -- data-done and data-tz -- and not the label, which does
+    still show the configured city as context and so differs between the two.
+    """
+    tampa = timezone_step(render(config_with(
+        location={"city": "Tampa", "state": "Florida", "country": "US"})))
+    seattle = timezone_step(render(config_with(
+        location={"city": "Seattle", "state": "Washington", "country": "US"})))
+    assert tampa == seattle
+
+
+def test_missing_timezone_leaves_the_step_open():
+    """Nothing saved means nothing to verify: the step stays unticked and the
+    JS bails on the empty value rather than comparing against ''."""
+    step = timezone_step(render(config_with(timezone="")))
+    assert 'data-tz=""' in step
+    assert 'data-done="0"' in step
+
+
+def test_zone_comparison_asks_for_the_time_of_day():
+    """Guard on the Intl options, which look like a stylistic choice.
+
+    dateStyle/timeStyle are late additions (Firefox shipped them in 91). An
+    implementation that does not know them ignores them and formats the date
+    alone -- which compares New York, Chicago and Madrid as equal and ticks
+    the step for a timezone that is plainly wrong. Explicit numeric fields
+    have been in Intl since ECMA-402 v1.
+    """
+    template = (PROJECT_ROOT / "web_interface" / "templates" / "v3"
+                / "partials" / "overview.html").read_text()
+    body = template[template.index("function sameZone"):]
+    body = body[:body.index("}())")]
+    # The comment above the options names dateStyle/timeStyle to explain why
+    # they are not used, so match on code only.
+    body = "\n".join(line for line in body.splitlines()
+                     if not line.lstrip().startswith("//"))
+    assert "dateStyle" not in body and "timeStyle" not in body, (
+        "zone comparison must not depend on dateStyle/timeStyle")
+    for field in ("hour:", "minute:", "year:", "month:", "day:"):
+        assert field in body, f"zone comparison dropped {field!r}"
+
+
+def test_zone_comparison_samples_both_sides_of_dst():
+    """One instant is not enough, and the shortfall is invisible for months.
+
+    America/New_York and America/Lima hold the same offset all winter, so a
+    check against now alone ticks the step in January for a panel that runs an
+    hour off from March. The comparison has to sample instants either side of
+    DST -- mid-January and mid-July, which covers both hemispheres.
+    """
+    template = (PROJECT_ROOT / "web_interface" / "templates" / "v3"
+                / "partials" / "overview.html").read_text()
+    body = template[template.index("function sameZone"):]
+    body = body[:body.index("}())")]
+    code = "\n".join(line for line in body.splitlines()
+                     if not line.lstrip().startswith("//"))
+    assert "Date.UTC" in code, (
+        "zone comparison samples only the current instant, so zones that "
+        "coincide seasonally would read as equal")
+    assert code.count("Date.UTC") >= 2, "expected an instant either side of DST"
+
+
+def _stamp(zone, instant):
+    """The JS comparison's algorithm, for pinning what it must decide.
+
+    There is no JS runtime here (and the repo has no JS test infra), so this
+    mirrors sameZone rather than executing it: same instants, same wall-clock
+    equality. It records the verdicts the shipped code has to reach.
+    """
+    from zoneinfo import ZoneInfo
+    return instant.astimezone(ZoneInfo(zone)).strftime("%m/%d/%Y %H:%M")
+
+
+@pytest.mark.parametrize(
+    "left,right,equivalent",
+    [
+        # Aliases: one zone under two names.
+        ("Asia/Calcutta", "Asia/Kolkata", True),
+        ("Europe/Kiev", "Europe/Kyiv", True),
+        # Same rules year-round: either renders the same times, so a panel set
+        # to one and browsed from the other is correctly configured.
+        ("America/New_York", "America/Toronto", True),
+        # Coincide in winter only -- the case a single-instant check gets wrong.
+        ("America/New_York", "America/Lima", False),
+        ("America/Phoenix", "America/Los_Angeles", False),
+        ("Australia/Sydney", "Pacific/Guadalcanal", False),
+        # Plainly different.
+        ("America/New_York", "America/Chicago", False),
+        ("America/New_York", "Europe/Madrid", False),
+    ],
+)
+def test_which_zone_pairs_must_count_as_the_same(left, right, equivalent):
+    from datetime import datetime
+    from zoneinfo import ZoneInfo
+
+    year = 2026
+    instants = [datetime(year, 1, 15, 12, tzinfo=ZoneInfo("UTC")),
+                datetime(year, 7, 15, 12, tzinfo=ZoneInfo("UTC"))]
+    matched = all(_stamp(left, at) == _stamp(right, at) for at in instants)
+    assert matched is equivalent, (
+        f"{left} vs {right}: sampling both seasons gave {matched}")
+
+
+@pytest.mark.parametrize(
+    "hardware,expected",
+    [
+        ({"rows": 32, "cols": 64, "chain_length": 2, "parallel": 1}, "1"),
+        ({"rows": 0, "cols": 0, "chain_length": 0, "parallel": 1}, "0"),
+    ],
+)
+def test_panel_size_step_still_reflects_config(hardware, expected):
+    """Regression guard: the hardware step is still decided server-side."""
+    config = config_with()
+    config["display"]["hardware"] = hardware
+    body = render(config)
+    match = re.search(r"<button[^>]*data-tab=\"display\"[^>]*>", body)
+    assert match, "panel-size step not found"
+    assert f'data-done="{expected}"' in match.group(0), match.group(0)

web_interface/templates/v3/partials/overview.html

@@ -63,13 +63,13 @@
 
 <!-- Getting Started checklist: non-gating, dismissible (localStorage), items
      auto-check from existing config/endpoints — no new persisted state.
-     Known heuristic limits (acceptable, disclosed): values left at legitimate
-     defaults (e.g. a user actually in Tampa) read as "not done". -->
+     The timezone step is verified against the browser's own zone rather than
+     compared to the shipped default; see the data-check="timezone" block below
+     for why. -->
 {% set _hw = main_config.display.hardware if main_config and main_config.display else {} %}
 {% set _hw_done = (_hw.rows or 0) > 0 and (_hw.cols or 0) > 0 and (_hw.chain_length or 0) > 0 %}
 {% set _loc = main_config.location if main_config and main_config.location else {} %}
-{% set _loc_done = (main_config.timezone and main_config.timezone != 'America/New_York')
-                   or (_loc.city and _loc.city != 'Tampa') %}
+{% set _tz = (main_config.timezone if main_config else '') or '' %}
 <div id="getting-started-card" class="bg-blue-50 border border-blue-200 rounded-lg p-4 mb-4" style="display:none" role="region" aria-label="Getting started checklist">
     <div class="flex items-start justify-between">
         <div class="flex-1">
@@ -78,8 +78,8 @@
             <ul class="space-y-1 text-sm" id="getting-started-items">
                 <li><button type="button" class="gs-item text-left w-full" data-done="{{ '1' if _hw_done else '0' }}" data-tab="display">
                     <i class="far fa-square mr-2"></i>Set your panel size (Display tab)</button></li>
-                <li><button type="button" class="gs-item text-left w-full" data-done="{{ '1' if _loc_done else '0' }}" data-tab="general">
-                    <i class="far fa-square mr-2"></i>Set your timezone and location (General tab)</button></li>
+                <li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="timezone" data-tz="{{ _tz }}" data-tab="general">
+                    <i class="far fa-square mr-2"></i>Set your timezone{% if _tz %} — currently {{ _tz }}{% if _loc.city %}, {{ _loc.city }}{% endif %}{% endif %} (General tab)<span data-gs-tz-note class="text-xs"></span></button></li>
                 <li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="installed" data-tab="plugins">
                     <i class="far fa-square mr-2"></i>Install a plugin from the Plugin Store</button></li>
                 <li><button type="button" class="gs-item text-left w-full" data-done="0" data-check="enabled" data-tab="plugins">
@@ -165,6 +165,91 @@
     });
     maybeAutoHide();
 
+    // Timezone: verified against the browser's own zone.
+    //
+    // This step used to tick when the saved timezone differed from the value
+    // config.template.json ships (America/New_York), with the saved city
+    // OR-ed in. Two things were wrong with that. "Differs from the default"
+    // answers "did somebody edit this?", but what the checklist needs to know
+    // is whether the value is RIGHT — so anyone who genuinely lives in the
+    // default zone could never satisfy it and the card nagged forever. And
+    // the city has no bearing on whether the timezone is set: because the two
+    // were OR-ed, saving a city ticked the step off with the timezone still
+    // wrong, which is the direction that actually breaks displays (event
+    // times render in the wrong zone).
+    //
+    // The browser already knows its zone, so compare against that: no new
+    // persisted state, no network, and it catches the reverse case too — a
+    // panel still set to the old zone after a move now stays unticked, where
+    // the old test ticked it the moment the value stopped being the default.
+    function sameZone(a, b) {
+        if (a === b) return true;
+        // Compare the wall-clock time each zone yields, not the identifiers:
+        // aliases (Asia/Calcutta vs Asia/Kolkata, Europe/Kiev vs Europe/Kyiv)
+        // name one zone and must not read as a mismatch.
+        //
+        // Sampled at three instants, all of which have to agree. Checking only
+        // now is not enough: America/New_York and America/Lima hold the same
+        // offset all winter, so a panel set to the wrong one of those would
+        // tick in January and then run an hour off from March. Mid-January and
+        // mid-July sit either side of DST in both hemispheres, so only zones
+        // that agree year-round match -- while Toronto still matches New York,
+        // which is right, since either renders the same times.
+        try {
+            var now = new Date();
+            var year = now.getUTCFullYear();
+            var instants = [now,
+                            new Date(Date.UTC(year, 0, 15, 12)),
+                            new Date(Date.UTC(year, 6, 15, 12))];
+            var stamp = function (tz, at) {
+                // Explicit numeric fields rather than dateStyle/timeStyle:
+                // those are late additions to Intl (Firefox shipped them in
+                // 91), and an implementation that does not know them ignores
+                // them and formats the date alone. That would compare
+                // New York, Chicago and Madrid as equal and tick the step for
+                // a timezone that is plainly wrong -- the exact failure this
+                // check exists to catch. These options have been in Intl
+                // since ECMA-402 v1.
+                return new Intl.DateTimeFormat('en-US', {
+                    timeZone: tz, year: 'numeric', month: '2-digit',
+                    day: '2-digit', hour: '2-digit', minute: '2-digit',
+                    hour12: false
+                }).format(at);
+            };
+            for (var i = 0; i < instants.length; i++) {
+                if (stamp(a, instants[i]) !== stamp(b, instants[i])) {
+                    return false;
+                }
+            }
+            return true;
+        } catch (e) {
+            // An unparseable zone in the config is worth surfacing, not hiding.
+            return false;
+        }
+    }
+
+    (function () {
+        var tzBtn = card.querySelector('[data-check="timezone"]');
+        if (!tzBtn) return;
+        var configured = tzBtn.dataset.tz || '';
+        if (!configured) return;          // nothing saved yet: leave it open
+        var local = '';
+        try {
+            local = (Intl.DateTimeFormat().resolvedOptions().timeZone) || '';
+        } catch (e) {
+            return;                        // no Intl: leave it to the manual tick
+        }
+        if (!local) return;
+        if (sameZone(configured, local)) {
+            markDone(tzBtn);
+            return;
+        }
+        // Unticked on its own says "wrong" without saying why; name the zone
+        // the browser is in so the step is actionable.
+        var note = tzBtn.querySelector('[data-gs-tz-note]');
+        if (note) note.textContent = ' — this browser is in ' + local;
+    }());
+
     // Plugin-derived states from the existing installed-plugins endpoint.
     fetch('/api/v3/plugins/installed')
         .then(function (r) { return r.json(); })

web_interface/templates/v3/partials/plugin_config.html

@@ -296,6 +296,26 @@
                 {% set enum_items = items_schema.get('enum') or [] %}
                 {% set x_options = prop.get('x-options') or {} %}
                 {% set labels = x_options.get('labels') or {} %}
+                {# A saved value that is no longer one of the options -- a team
+                   code the league retired, an option dropped from the schema --
+                   has no checkbox to render, so it would sit unseen in the
+                   hidden input below and be posted back on save. The schema
+                   rejects it and the save endpoint returns 400, which blocks
+                   editing any other field on the plugin until the stale entry
+                   is found and removed. Drop them here instead, and say which,
+                   so the value is not lost silently. Only when the widget
+                   actually has options: an empty enum means nothing to check
+                   against, and filtering on it would wipe the field. #}
+                {% set stale_values = (array_value | reject('in', enum_items) | list) if enum_items else [] %}
+                {% set array_value = (array_value | select('in', enum_items) | list) if enum_items else array_value %}
+
+                {% if stale_values %}
+                <div class="mt-1 mb-2 rounded border border-amber-300 bg-amber-50 px-3 py-2 text-sm text-amber-800"
+                     data-stale-options="{{ field_id }}">
+                    No longer offered, and will be removed when you save:
+                    <span class="font-mono">{{ stale_values | join(', ') }}</span>.
+                </div>
+                {% endif %}
 
                 <div class="mt-1 space-y-2">
                     {% for option in enum_items %}