Scrolling ran at 44-46 fps on a 2x128x64 chain and 14-17% of frames took 41-53ms, which reads as judder. Four independent causes, each measured on the hardware; details and the diagnostic recipe are in docs/SCROLL_PERFORMANCE.md. The high-FPS loop slept a flat 8ms after every render. display() has already blocked on the panel's vsync by then, so that sleep was added to a wait that had happened: ~4ms of render plus 8ms put each iteration at ~12ms against a 10ms refresh grid, so every swap missed a refresh and the loop settled at 50fps while asking for 125 -- with no headroom, so a further 14% of frames slipped again. It now sleeps only the remainder, with a 1ms floor so plugin threads still get the GIL. ScrollHelper stepped position on a wall clock at 1/scroll_delay steps per second. Plugins set scroll_delay to the frame period, so that comparison sat exactly on its own threshold: a frame arriving a hair early moved zero pixels and rendered an identical frame, dirty-tracking skipped the swap, it returned in ~2ms, and the beat repeated. No scroll_delay value tunes that out -- a shorter delay trades stalled frames for periodic double-steps. Both modes now accumulate elapsed time at the same configured speed, so position stays proportional to real time. Sub-pixel blending goes back to off by default. It renders a half-step by mixing two adjacent columns, which on a coarse panel showing pixel-font text alternates crisp and smeared frames and reads as shimmer -- visibly worse than integer stepping on the hardware. Vegas mode still opts in. disk_cache uses orjson when importable, falling back to the stdlib. Encoding a ~1MB record drops from 14.8ms to 5.4ms end-to-end, and that work holds the GIL while a marquee is on screen. display_manager also checksummed the whole framebuffer twice per frame (dirty tracking, then the preview snapshot); the snapshot now takes the checksum the caller already computed. New src/common/scroll_config.py resolves scroll settings in one place. Five ticker plugins each hand-rolled this and disagreed: odds-ticker ranked the deprecated scroll_pixels_per_second above the documented scroll_speed/delay pair, and because that key carries a schema default the documented settings were dead for every user (ChuckBuilds/ledmatrix-plugins#408), while ledmatrix-leaderboard read the same key only as a fallback. The resolver also warns when a speed will not advance a whole number of pixels per refresh, which is the property that actually determines whether a scroll looks smooth. scripts/build_rgbmatrix_nogil.sh rebuilds the rgbmatrix binding so it releases the GIL. Upstream declares SwapOnVSync without nogil, unlike SetPixel/Clear/Fill beside it, so the render thread held the GIL for the whole vsync wait and starved background threads into long uninterruptible bursts. The script patches, builds and self-verifies into a scratch tree; --install backs up the original and rolls back if the service does not come back healthy. Measured after: 100 fps locked, no stalls observed, render thread down from 51% to 19% of one core. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
7.3 KiB
Scroll Performance
How scrolling is paced on this hardware, what was wrong with it, and how to configure a plugin so its marquee is smooth.
Measured on a Raspberry Pi 4 driving a 2×128×64 chain (256×64 logical) at
limit_refresh_rate_hz: 100. Numbers below come from that panel.
| before | after | |
|---|---|---|
| scroll frame rate | 44–46 fps | 100 fps, locked |
| frames ≥ 45 ms | 14–17% | none observed |
| dominant frame time | 20 ms | 10 ms |
| disk cache write (~1 MB) | 14.8 ms | 5.4 ms |
The one rule that matters
Motion is smooth when the strip advances a whole number of pixels per panel refresh.
On a 100 Hz panel the crisp speeds are 100 px/s, 200 px/s, 300 px/s. A speed that does not divide evenly has to do one of two bad things:
- blend two adjacent columns to render a half-step — on pixel-font text this alternates crisp and smeared frames and reads as shimmer, or as the text jumping a pixel ahead of itself;
- repeat a frame — the strip stands still, then jumps, which reads as judder.
Neither is tunable away. Pick a speed that divides evenly.
src.common.scroll_config.resolve() warns when a configured speed will not,
and names the nearest speed that will.
Configuring a plugin
Use the shared resolver rather than reading config keys yourself:
from src.common import scroll_config
settings = scroll_config.configure(
self.scroll_helper,
plugin_config=self.config,
global_config=self.global_config,
refresh_hz=scroll_config.refresh_hz_from_config(self.global_config),
plugin_logger=self.logger,
)
It resolves every config shape in one place, applies the speed, and returns what it did. Precedence, highest first:
display_options.scroll_speed+scroll_delay— the recommended formdisplay.scroll_speed+scroll_delay— deprecated shapescroll_speed+scroll_delayat the root — legacy flatscroll_pixels_per_second— deprecated- the global
displayblock - the built-in default (100 px/s)
scroll_speed is pixels per frame and scroll_delay is the frame period in
seconds, so the pair means scroll_speed / scroll_delay px/s. The recommended
config for a 100 Hz panel:
"display_options": { "scroll_speed": 1.0, "scroll_delay": 0.01 }
Why the deprecated key ranks below the explicit pair
Because some plugins give scroll_pixels_per_second a schema default, and
schema defaults are merged into plugin config. Ranking it above the pair means
it is always present and always wins, so the documented settings become
unreachable. That is a real, shipped bug — see
ledmatrix-plugins#408.
If you are writing a plugin: do not give a deprecated key a schema default.
What was actually wrong
Four independent faults, each found by measurement.
1. The frame loop slept on top of a wait it had already done
display_controller.py ran the high-FPS loop as render → SwapOnVSync (blocks to the panel's refresh) → time.sleep(0.008) → plugin ticks. The sleep was
unconditional and added to a wait that had already happened. Render work
measured ~4 ms, so each iteration cost ~12 ms against a 10 ms refresh grid —
every swap missed a refresh and landed on the next one. The loop settled at
exactly 50 fps while asking for 125, with no headroom, so ~14% of frames
slipped a further refresh.
Now the loop sleeps only the remainder of the frame budget, with a 1 ms floor so plugin threads still get the GIL.
2. SwapOnVSync held the GIL while blocking
The rgbmatrix binding declares it without nogil (unlike SetPixel, Clear
and Fill immediately above it in cppinc.pxd), so the render thread held the
GIL for the entire vsync wait — most of every frame. Background threads were
starved into long uninterruptible bursts; a 1.5 MB API response costs ~17 ms to
parse and ~18 ms to re-encode for the cache, and json.raw_decode cannot be
preempted mid-document. Those bursts are what the render loop then waited on.
Fixed by rebuilding the binding: scripts/build_rgbmatrix_nogil.sh.
3. Sub-pixel blending was wrong for this display
Enabling it made things worse, not better — see the rule at the top. It is off
by default and only Vegas mode opts in via set_sub_pixel_scrolling(True).
4. Frame-based stepping raced the vsync clock
Frame-based mode gated motion on a wall clock at 1/scroll_delay steps per
second. Plugins set scroll_delay to the frame period, which puts that
comparison exactly on its own threshold: a frame arriving a hair early moved
zero pixels and rendered an identical frame, which dirty-tracking skipped, so
it returned in ~2 ms and the beat repeated. No scroll_delay value tunes this
out — a shorter delay just trades stalled frames for periodic double-steps.
ScrollHelper now accumulates elapsed time in both modes at the same
configured speed, so position stays proportional to real time.
Diagnosing a juddery scroller
Avg FPS will lie to you. It is a 100-frame moving average, and a 2 ms
duplicate frame plus a 21 ms double-wait average to exactly 10 ms. A ticker
that is stalling on half its frames still reports a healthy 100.0.
Look at the distribution instead:
journalctl -u ledmatrix --since "-10min" --no-pager \
| grep -oE "Frame time: [0-9.]+ms" | awk '{print $3}' | sed 's/ms//' \
| awk '{printf "%.0f\n", $1}' | sort -n | uniq -c
Reading it, on a 100 Hz panel:
| you see | it means |
|---|---|
| everything at 10 ms | healthy |
| a mode at ~2 ms | duplicate frames — the swap was skipped because the image did not change. The scroller is advancing less than one pixel per frame. |
| a mode at 20/30/50 ms | frames missing refreshes — per-frame work is overrunning, or a background thread is holding the GIL |
Avg FPS above 100 |
duplicates present, unless the scroll cycle has completed and is idling |
Then confirm what the plugin actually loaded — config edits do not always reach the running code:
journalctl -u ledmatrix --since "-5min" --no-pager | grep -iE "px/s|px/frame"
If a plugin logs its scroll config twice with different modes, the second line is what is running.
Rebuilding the binding
bash scripts/build_rgbmatrix_nogil.sh # build into a scratch dir
sudo bash scripts/build_rgbmatrix_nogil.sh --install
sudo bash scripts/build_rgbmatrix_nogil.sh --rollback
The build never touches the installed module. --install backs up the original
to ~/rgbmatrix-core.so.ORIGINAL first, and rolls back automatically if the
service does not come back healthy. Requires build-essential; Cython is
installed into a cached venv under ~/.cache/ledmatrix-cython.
Re-run it after upgrading rpi-rgb-led-matrix, since a library upgrade
replaces the patched binding.
Faster JSON
src/cache/disk_cache.py uses orjson when it is importable and falls back to
the stdlib otherwise, so it is optional:
sudo pip3 install --break-system-packages orjson
Encoding is where it pays — about 7× on this hardware. Decoding gains far less
(~1.3× on large payloads) because the cost there is building Python objects,
not scanning text. That is also why moving parsing to a subprocess does not
help: pickle.loads of the same payload costs 8.1 ms against json.loads at
10.9 ms, so the work just moves rather than disappearing.