# Offscreen Rendering **Status: proposed (2026-09-24).** Nothing here is implemented yet. This is the design for review. When it lands, this file becomes the reference for how plugin content is rendered off the render thread. ## The problem Vegas mode builds its ticker from every plugin's content. Most of that work already happens on a background prefetch thread (`RenderPipeline.start_prefetch`). But any plugin whose content needs the **shared display canvas** is deferred to the render thread (`RenderPipeline.drain_deferred`), one plugin every two seconds. The code's own comments put each of those at 40–600 ms, and the render thread presents no frames while one runs. On hdpi (Pi 4, 512×64) most plugins take that path: geochron, tide-display, news, hockey-scoreboard, ledmatrix-stocks, incoming-packages, clock-simple, countdown, birdnet-go, ledmatrix-music and odds-ticker. They arrive in bursts ("Whole group deferred; strip will extend as it drains") every minute or so, 12 fetches in five minutes. That is the "occasional pause" a viewer sees. An 8-minute soak (`scripts/frame_soak.py --preview`) of the #628 build on hdpi: | late by | frames | |---|---| | 1 refresh | 238 | | 2 | 32 | | 3–5 | 30 | | 6+ | 5 | | freezes ≥ 250 ms | 2 (0.97 s total) | The 3+ rows and the freezes are the pauses. The single-refresh row is a separate problem: the blit is 6 ms of a 10 ms refresh, so there is little slack. It is covered under *What this does not fix*. ## Why a plugin is canvas-bound The plugin-facing canvas is a set of shared attributes on `DisplayManager`: `image`, `draw`, `matrix`, and the `width`/`height` properties that read from `matrix`. Three adapter paths (`src/vegas_mode/plugin_adapter.py`) need them, and each returns `None` under `offscreen_only=True` so the plugin is queued for the render thread: 1. **Display capture** (`_capture_display_content`): clear the canvas, call `plugin.display()`, copy `display_manager.image`. Used by any plugin without `get_vegas_content()` or a populated `scroll_helper`. 2. **Scroll-content generation** (`_trigger_scroll_content_generation`): a ticker plugin whose `scroll_helper.cached_image` is empty is made to build it by calling `display(force_clear=True)` or `_create_scrolling_display()`. Both draw on the canvas. 3. **Narrowed rendering** (`DisplayManager.render_size`): swaps the shared `matrix`, `image` and `draw` for a narrower set so the plugin lays out for `render_width_pct`. The render thread would see the swap mid-frame. The render thread keeps the canvas coherent only because nothing else touches it at the same time. A background thread can't use it. ## The design: a per-thread render target `capture_mode()` is already per-thread (#423 made its state a `threading.local`, so a background capture no longer suppresses the render loop's pushes). The same move applies to the canvas itself: ```python with display_manager.offscreen(width=None, height=None) as surface: plugin.display(force_clear=True) content = surface.image.copy() ``` For the **calling thread only**, inside the block: | accessor | resolves to | |---|---| | `display_manager.image`, `.draw` | the surface's own image and draw: a fresh black canvas, `fontmode = "1"` | | `display_manager.matrix` | a logical proxy reporting the surface size, so `width`/`height` and plugins that read `matrix.width` follow it. Hardware calls through it (`SetImage`, `SwapOnVSync`, `Clear`, brightness writes) are inert. | | `update_display()`, `clear()` | canvas-only: the block implies capture mode, which is already per-thread | | `set_scrolling_state()`, `set_frame_hold()` | no-ops, so a plugin's `display()` cannot re-pace the live scroll. Today it can, when it is captured on the render thread. | Every other thread sees the real canvas, unchanged. The render loop in particular keeps presenting while a plugin draws elsewhere. ### Implementation sketch - `image`, `draw` and `matrix` become properties over `_image`, `_draw` and `_matrix`, plus a thread-local current surface. The getter returns the surface's value when the calling thread has one, else the shared one; setters mirror that. That costs about 0.1 µs per access, and `update_display()` reads each a handful of times per frame. Every existing `self.image = ...` in `DisplayManager` (`clear()`, setup, fallback) keeps working and becomes thread-correct for free. - `render_size()` is rebuilt on `offscreen()`: it creates or narrows the calling thread's surface instead of swapping shared state. - `offscreen()` nests and always restores on exit, including when the plugin raises. - `VisualDisplayManager` (the plugin test harness) gets the same method, for parity. ### Adapter changes - `get_content(offscreen_only=True)` stops returning `None` for the three paths above. Each runs inside `display_manager.offscreen(render_width)`. - `_capture_display_content` and `_trigger_scroll_content_generation` drop their "copy the shared image, restore it afterwards" bookkeeping, since the shared image is never touched. - **Take the plugin's lock.** `PluginManager.get_plugin_lock()` keeps `update()` and `display()` mutually exclusive in normal rotation, but Vegas never takes it, so today's render-thread captures already race `update()`. Off the render thread the adapter can afford to wait: blocking acquire with a timeout (proposed 2 s). On timeout it keeps the cached segment and tries again next group. - `drain_deferred()` and the deferred queue are deleted. The only render-thread fetch left is the inline fallback when no prepared group is ready, which in practice is the first extension. Prefetching at start removes that too. ## Risks, and what was checked 1. **Plugins holding their own reference to the shared `draw` or `image`.** They would keep drawing into the shared canvas, and routing by thread can't redirect them. A grep of the 49 plugins installed on hdpi found none storing `display_manager.draw` or `.image` in an attribute (a pattern search, so indirect aliasing would slip past it). A plugin that did would draw into an image nobody displays, which trims to a blank segment. That is not corruption, and it is no worse than today. 2. **Plugins calling the matrix directly.** None in the audit. Inside `offscreen()` the proxy makes it inert anyway. 3. **Font thread-safety.** `FontManager` shares font objects across plugins. Measured on Pillow 12.3, two threads rendering text take 1.94× as long as one, so text rendering holds the GIL and FreeType is never entered concurrently. Re-check if Pillow changes that. 4. **Plugin thread-safety.** `display()` moves to the prefetch thread. The plugin lock makes it exclusive with `update()`, which is more protection than it has today. Threads a plugin starts itself are not covered, as today. 5. **The GIL.** Moving 40–600 ms of plugin rendering off the render thread removes the pauses, but the work still needs the GIL. Pillow drawing holds it, and a waiting thread only gets it back after the switch interval (default 5 ms). Expect some single-refresh late frames while a prefetch runs. Measure with the soak. Lowering `sys.setswitchinterval` during Vegas (e.g. 1 ms) is a one-line experiment, and a render process separate from plugin work is the structural answer (the "native presenter" step). ## What this does not fix - **The blit.** Copying a 512×64 frame into the matrix (`SetImage`) is ~6 ms at 8 PWM bits on a Pi 4, leaving ~4 ms of slack per refresh. That is the main source of the single-refresh late frames. Holding frames for two refreshes (≈50 px/s) doubles the budget. Cutting the blit itself is the native-presenter step. - **Appending to the strip.** `append_content()` rebuilds the whole strip (8,000–20,000 px wide, 1.5–3.8 MB) on the render thread for every appended block. Not yet measured in isolation. Candidates: build the extended strip on the prefetch thread and swap one reference, or keep the strip as a list of chunks so an append is O(block). Measure first. - **Live refreshes pushed from `update()`.** Some sports plugins call `display()` and `update_display()` from inside `update()`, which runs on the update worker and can push to the panel mid-Vegas. That is a separate hazard. `offscreen()` gives a tool for it (run the update worker offscreen while Vegas owns the panel), but it is out of scope here. ## Test plan - **Unit, `DisplayManager`:** one thread inside `offscreen()` draws while another reads `image`/`draw`/`matrix`/`width`/`height` and sees the real canvas. Also: `update_display()` and `set_scrolling_state()` are inert inside; `render_size()` narrows only the calling thread; nesting and exceptions restore state. - **Unit, adapter:** a stub display-capture plugin and a stub scroll-helper plugin both return content with `offscreen_only=True`, and nothing is queued for the render thread. The plugin lock is taken, and a timeout keeps the cached segment. - **Emulator integration:** a stub canvas-bound plugin whose `display()` sleeps 300 ms. The Vegas render loop never goes a frame without presenting (frame timing recorder: zero freezes). - **Hardware:** an hdpi soak, A/B against the #628 build, alternating order. Targets: no freezes, an empty 6+ bucket, the 3–5 bucket near zero, and the late rate below 0.66%. ## Rollout `display.vegas_scroll.offscreen_prefetch` (default `true`) restores today's deferred path when `false`. Keep it for one release, then delete it along with the deferred path. ## Open questions 1. Keep the kill switch, or ship without one? 2. Plugin lock timeout: skip the plugin and keep its cached segment (proposed), or wait longer? 3. Strip appends: in this change, or measured and handled separately (proposed)?