feat(scroll): report a panel that cannot reach its refresh cap, and suggest one it can hold (#759)

* feat(scroll): report a panel that cannot reach its refresh cap, and suggest one it can hold

Scroll speeds are solved against display.hardware.limit_refresh_rate_hz,
which is only a ceiling. A panel that cannot reach it still moves whole
pixels per frame, but every scroll runs slow by the shortfall and the
"smooth" ladder is the cap's, not the panel's. A user rig (Pi 4, 2x128x64,
adafruit-hat-pwm, pwm_bits 9, gpio_slowdown 5) measured 107.6-113.1 Hz under
a 120 Hz cap: 60 px/s ran at 55, and nothing said why.

- scroll_config: refresh_shortfall() (more than 3% under the planned rate),
  holdable_cap() (a multiple of 10, 5% under the measurement, since the
  measurement is the fast end of an uncapped panel's drift), and
  describe_refresh_shortfall().
- FrameTimingRecorder.plan_refresh(): once the measured period has held for
  three trusted windows, a shortfall is logged once as a warning naming the
  cap to use. DisplayManager calls it only for a real panel, not the
  emulator or the fallback canvas. The stats file records
  planned_refresh_hz (additive).
- GET /api/v3/config/refresh-rate, plus a hint under the Display tab's
  Limit Refresh Rate field (js/pages/display.js) with a button that fills in
  the suggested cap.
- _panel_refresh_hz (behind the Vegas slider's advice) ignores a measurement
  written under a different cap, so a changed cap stops being advised from
  the old rate before the display restarts.

Verified on ledpi with a temporary 200 Hz cap: the warning logged about a
minute after the restart ("about 132 Hz ... Set Limit Refresh Rate to
120 Hz"), the endpoint returned the same shortfall, and the Display tab
showed the hint; its button filled in 120. ledpi was restored afterwards.
Rebased onto main after the Display tab became an ES-module page (#771); the
hint moved from inline script into display.js, with a jsdom test.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(scroll): count only agreeing windows toward the refresh shortfall; ignore a null planned rate

Review fixes on #759.

- A window the period estimate rejects (more than MAX_REFRESH_DROP faster
  than the adopted period) no longer counts toward the shortfall check, and
  it restarts the run. After a loaded start fixed a slow period, later
  windows at the real, faster rate were rejected yet still counted, so the
  warning could name the slow rate against a cap the panel was meeting. It
  now needs REFRESH_CHECK_WINDOWS consecutive windows that agree with the
  period.
- _panel_refresh_hz treats a stats file whose planned_refresh_hz key is
  present but null as no measurement. The emulator and the fallback canvas
  write it that way (DisplayManager never plans a refresh for them), and
  their frame rate says nothing about the cap. A file with no such key (an
  older display) keeps the old behaviour.

Three new tests fail on the previous code and pass now.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

src/common/frame_timing.py

@@ -135,6 +135,8 @@ import time
 import traceback
 from typing import Any, Callable, Dict, List, Optional, Tuple, TypedDict
 
+from src.common import scroll_config
+
 logger = logging.getLogger(__name__)
 
 #: Bumped when a field changes meaning, so a reader can refuse stale files.
@@ -179,6 +181,12 @@ MAX_REFRESH_DROP = 0.2
 #: trusted -- about a second of scrolling.
 MIN_FRAMES_FOR_REFRESH = 90
 
+#: Consecutive windows that agree with the adopted refresh period (the one
+#: that adopted it counts) before a panel slower than its cap is reported. The
+#: estimate can still fall by up to MAX_REFRESH_DROP per window early on; the
+#: warning should not name a rate one more window would have corrected.
+REFRESH_CHECK_WINDOWS = 3
+
 FLUSH_INTERVAL = 10.0
 
 #: A scroll's last frame older than this is a stall worth a stack dump.
@@ -441,6 +449,12 @@ class FrameTimingRecorder:
             1.0 / refresh_hz if refresh_hz and refresh_hz > 0 else None)
         # The first estimate, until a second window agrees with it.
         self._refresh_candidate: Optional[float] = None
+        # The rate scroll speeds are solved against; see plan_refresh().
+        self.planned_refresh_hz: Optional[float] = None
+        # Trusted windows seen since the period was adopted, until the
+        # shortfall check has run.
+        self._refresh_windows = 0
+        self._shortfall_checked = True
         self.totals: Dict[str, Any] = {
             "static_frames": 0,
             "scroll_frames": 0,
@@ -639,7 +653,20 @@ class FrameTimingRecorder:
                     self._refresh_candidate = estimate
             elif current * (1.0 - MAX_REFRESH_DROP) <= estimate < current:
                 self.refresh_period = estimate
+            # Only a run of windows that agree with the period counts toward
+            # the shortfall check. One that disagrees (a faster one than the
+            # period may fall to, say, after a loaded start fixed a slow one)
+            # was rejected above, so the period does not reflect it, and a
+            # warning built on that period would name a rate the panel is not
+            # at. It resets the run; the check then waits for three that agree.
+            current = self.refresh_period
+            if current is not None:
+                agrees = abs(estimate - current) <= current * MAX_REFRESH_DROP
+                self._refresh_windows = self._refresh_windows + 1 if agrees else 0
         period = self.refresh_period
+        if (period and not self._shortfall_checked
+                and self._refresh_windows >= REFRESH_CHECK_WINDOWS):
+            self._check_refresh_shortfall(1.0 / period)
 
         histograms = self.histograms
         for frame in batch:
@@ -688,6 +715,25 @@ class FrameTimingRecorder:
                 elif missed <= -1:
                     totals["early_frames"] += 1
 
+    def plan_refresh(self, hz: Optional[float]) -> None:
+        """Say what rate scroll speeds are solved against, before frames arrive.
+
+        ``DisplayManager.refresh_hz``: the configured cap. Once the measured
+        rate has held for :data:`REFRESH_CHECK_WINDOWS` windows in a row, a panel that
+        falls short of it is logged once, with a cap it can hold (see
+        :func:`src.common.scroll_config.refresh_shortfall`). The display
+        manager calls this only for a real panel.
+        """
+        self.planned_refresh_hz = hz
+        self._shortfall_checked = not hz
+
+    def _check_refresh_shortfall(self, measured_hz: float) -> None:
+        """Log, once, a panel that cannot reach the rate speeds assume."""
+        self._shortfall_checked = True
+        shortfall = scroll_config.refresh_shortfall(measured_hz, self.planned_refresh_hz)
+        if shortfall:
+            logger.warning(scroll_config.describe_refresh_shortfall(shortfall))
+
     def snapshot(self) -> Dict[str, Any]:
         """The JSON document: cumulative since this process started."""
         if not self._binding_checked:
@@ -704,6 +750,9 @@ class FrameTimingRecorder:
             "bucket_ms": BUCKET_MS,
             "freeze_seconds": FREEZE_SECONDS,
             "measured_refresh_hz": round(1.0 / period, 2) if period else None,
+            # Additive: what scroll speeds were solved against, so a reader can
+            # tell a stale file (written under another cap) from this one.
+            "planned_refresh_hz": self.planned_refresh_hz,
             "binding_releases_gil": self._binding_gil,
             "info": info,
             "totals": copy.deepcopy(self.totals),

src/common/scroll_config.py

@@ -538,3 +538,66 @@ def speed_advice(
         "smooth": smooth,
         "alternatives": [as_dict(c) for c in alternatives],
     }
+
+
+#: A measured refresh this far below the rate speeds are planned for means
+#: the panel cannot reach its cap. Smaller gaps are the cap's own slack and
+#: the estimate's: one rig measured 99.95 Hz under a 100 Hz cap.
+REFRESH_SHORTFALL = 0.03
+
+#: How far under the measured rate a suggested cap sits. The measurement is
+#: the fast end of the panel's refreshes (frame_timing takes the 10th
+#: percentile of intervals), and an uncapped panel drifts: one read
+#: 107.6-113.1 Hz over 15 seconds. A cap inside that band would not hold.
+CAP_HEADROOM = 0.05
+
+
+def holdable_cap(measured_hz: Any) -> Optional[int]:
+    """A refresh cap the panel can hold: a multiple of 10, 5% under what it measured.
+
+    A multiple of 10 because its whole-pixel speeds are round numbers (a
+    100 Hz cap gives 50 and 100 px/s). None without a usable measurement, or
+    when the panel is too slow for any cap of 10 Hz or more.
+    """
+    hz = _coerce(measured_hz)
+    if hz is None:
+        return None
+    cap = int(hz * (1.0 - CAP_HEADROOM) // 10) * 10
+    return cap if cap >= 10 else None
+
+
+def refresh_shortfall(measured_hz: Any, planned_hz: Any) -> Optional[Dict[str, Any]]:
+    """When the panel refreshes measurably slower than speeds are planned for.
+
+    ``planned_hz`` is what :func:`configure` solves against -- the
+    ``limit_refresh_rate_hz`` cap, or :data:`DEFAULT_REFRESH_HZ` when it is 0.
+    A panel that cannot reach it still moves whole pixels per frame, but every
+    speed runs slow by the shortfall and the ladder of smooth speeds is the
+    cap's, not the panel's. None when there is no measurement, or the panel
+    reaches the cap (or beats it, as some do by a few Hz).
+    """
+    measured, planned = _coerce(measured_hz), _coerce(planned_hz)
+    if measured is None or planned is None:
+        return None
+    if measured >= planned * (1.0 - REFRESH_SHORTFALL):
+        return None
+    return {
+        "measured_hz": round(measured, 1),
+        "planned_hz": round(planned, 1),
+        "suggested_cap_hz": holdable_cap(measured),
+        "slow_percent": round((1.0 - measured / planned) * 100),
+    }
+
+
+def describe_refresh_shortfall(shortfall: Dict[str, Any]) -> str:
+    """One log line for :func:`refresh_shortfall`'s answer."""
+    text = (
+        f"The panel refreshes at about {shortfall['measured_hz']:.0f} Hz, below "
+        f"the {shortfall['planned_hz']:.0f} Hz that scroll speeds are planned "
+        f"for (display.hardware.limit_refresh_rate_hz), so every scroll runs "
+        f"about {shortfall['slow_percent']}% slower than configured and the "
+        f"smooth speeds are worked out for a rate this panel never reaches.")
+    if shortfall.get("suggested_cap_hz"):
+        text += (f" Set Limit Refresh Rate to {shortfall['suggested_cap_hz']} Hz "
+                 f"(web UI, Display tab), which this panel can hold, and restart.")
+    return text

src/display_manager.py

@@ -393,6 +393,11 @@ class DisplayManager:
         
         self._setup_matrix()
         logger.info("Matrix setup completed in %.3f seconds", time.time() - start_time)
+        # Only a real panel's swaps wait on its refresh: the emulator and the
+        # fallback canvas pace themselves, so "slower than the cap" would be
+        # noise there.
+        if self.matrix is not None and os.environ.get('EMULATOR', 'false') != 'true':
+            self.frame_timing.plan_refresh(self.refresh_hz)
         self._setup_scan_order_compensation()
         
         font_time = time.time()
@@ -1501,7 +1506,9 @@ class DisplayManager:
         fractional-pixel motion. See src/common/scroll_config.py.
 
         Note this is the configured *cap*, not necessarily what the panel
-        achieves -- scripts/scroll_speeds.py --measure reports the real rate.
+        achieves -- scripts/scroll_speeds.py --measure reports the real rate,
+        and the frame-timing recorder logs a warning, with a cap the panel can
+        hold, once it has measured a panel that falls short of this.
         """
         hardware = (self.config.get('display') or {}).get('hardware') or {}
         try: