Cursor-following auto-zoom on Linux Wayland: reading the cursor from the GPU in Rust

Screen Studio on macOS made one idea popular: while you record, the video quietly zooms in on whatever you're working on, then glides back out when you move away. On Linux Wayland, that one feature is surprisingly hard. The reason has nothing to do with video. Wayland won't tell an app where the cursor is.

This post covers how I got around that in HyperGSR, an MIT-licensed screen recorder written in Rust. It covers the trick (read the cursor from the GPU's hardware cursor plane), the camera (a critically damped spring), and what it costs (about 56 MB of RAM at 1080p60).

You can try the camera in your browser before reading any code: the HyperGSR landing page runs the same spring model.

KunalSiyag / gpu-recorder-rs

Screen recorder for Linux Wayland, written in Rust, whose camera follows your cursor. Rootless DRM cursor tracking, spring-physics auto-zoom, NVENC/VAAPI encoding.

Website · Download v1.1.0 · Promo video (10 s, 1080p60) · How it works · Write-up on DEV · Contributing

HyperGSR (gpu-recorder-rs) is a screen recorder for Linux, written in Rust. It reads the cursor straight from the GPU's hardware cursor plane, without root, and moves a critically damped spring camera to wherever you are working, the way Screen Studio does on macOS. Capture goes through PipeWire and the XDG desktop portal, and encoding runs on NVENC or VAAPI.

It runs on native Wayland (COSMIC, GNOME, KDE Plasma, Hyprland, Sway) and on X11.

Contents

• What it does
• Screenshots
• Measured footprint
• Install
• Usage
• Zoom triggers
• How it works
• Compared with other recorders
• Project layout
• Roadmap
• Contributing

What it does

• Tracks the cursor on Wayland without root. Recorders that call XQueryPointer go blind or stall once a native Wayland window has focus. HyperGSR opens /dev/dri/card* through libdrm and reads CRTC_X…

View on GitHub

The problem: there is no global cursor on Wayland

On X11 you call XQueryPointer and get global coordinates. Wayland deliberately doesn't offer that. A client only learns where the pointer is while the pointer is over the client's own surface. That's a good security property, and it's the whole reason auto-zoom recorders are rare on Linux.

The usual workarounds each break somewhere:

I started by reverse-engineering gpu-screen-recorder 6.1.4 to see how an existing GPU recorder handles this. It uses X11 for the cursor too. Along the way I found a hardcoded /var/lib/flatpak path that breaks user-level Flatpak installs. The notes are in RE_ANALYSIS.md.

The trick: ask the display hardware

Every modern GPU has a small hardware cursor plane: a dedicated overlay the display controller composites on top of the frame. The compositor has to tell the kernel where to draw it, and those values are exposed as DRM plane properties: CRTC_X, CRTC_Y, and FB_ID (the cursor image).

Reading those properties doesn't require becoming DRM master, and it doesn't change any modes. The logged-in desktop user can already open /dev/dri/card*. So HyperGSR loads libdrm, turns on universal planes and atomic properties, finds the cursor plane (plane #92 on my machine), and reads it:

// src/cursor.rs — DrmCursorReader::query_cursor()
for j in 0..pcount {
    let prop_id = *(*props).props.add(j);
    let pval = *(*props).prop_values.add(j);
    let pr = (self.get_prop)(self.fd, prop_id);
    if !pr.is_null() {
        let name_bytes =
            std::slice::from_raw_parts((*pr).name.as_ptr() as *const u8, 32);
        if name_bytes.starts_with(b"FB_ID\0") {
            fb_id = pval;
        } else if name_bytes.starts_with(b"CRTC_X\0") {
            crtc_x = Some((pval as u32) as i32);
        } else if name_bytes.starts_with(b"CRTC_Y\0") {
            crtc_y = Some((pval as u32) as i32);
        }
        (self.free_prop)(pr);
    }
}

A non-zero FB_ID means the cursor is visible, and a change in FB_ID means the cursor shape changed (arrow to text beam, for example). That's a free signal that you've moved onto something interactive. The GUI polls this every 16 ms, which works out to about 60 Hz.

Because XWayland can block, the tracker never even opens an X11 connection when the DRM plane is available:

// src/cursor.rs — CursorTracker::new()
let drm_cursor = DrmCursorReader::try_open();
let x11_conn = if drm_cursor.is_some() {
    None
} else {
    X11CursorConn::try_open()
};

Caveats, honestly:

• If your compositor draws the cursor in software (some VMs, some drivers), there is no plane to read. HyperGSR then falls back to XQueryPointer. That's exact on Xorg, but on Wayland it only follows the cursor over XWayland windows.
• I develop and test on COSMIC with a hybrid AMD + NVIDIA laptop. GNOME, KDE Plasma, Hyprland and Sway should behave the same, since this is kernel-level, but I haven't verified them. Reports are welcome.

The camera: a spring with exactly enough damping

Knowing where the cursor is gets you halfway. The other half is moving the virtual camera so it feels filmed rather than scripted. Linear interpolation looks robotic, and an underdamped spring wobbles. The sweet spot is a critically damped spring, where the damping coefficient is 2√k. It reaches the target as fast as possible without overshooting.

The camera has three springs: pan X, pan Y, and zoom. Pan runs at 78% of the zoom stiffness, so the frame drifts slightly behind the zoom like a real fluid head:

// src/zoom_engine.rs — VirtualCamera::step()
let k_zoom = cfg.spring_stiffness.clamp(20.0, 260.0);
let c_zoom = 2.0 * k_zoom.sqrt();
let k_pan = (k_zoom * 0.78).clamp(18.0, 220.0);
let c_pan = 2.0 * k_pan.sqrt();

let ax = -k_pan * (self.cam_x - self.target_x) - c_pan * self.vel_x;
let ay = -k_pan * (self.cam_y - self.target_y) - c_pan * self.vel_y;
let az = -k_zoom * (self.cam_zoom - self.target_zoom) - c_zoom * self.vel_zoom;

self.vel_x += ax * dt;
self.cam_x += self.vel_x * dt;
// ...same for y and zoom, then clamp the centre so the crop
// never samples outside the [0, 1] frame

If you want the math behind this, Ryan Juckett's Damped Springs and Daniel Holden's Spring-It-On are the two write-ups I keep going back to.

Deciding when to zoom

The spring decides how the camera moves. A separate rule decides where it's going. Zooming whenever the cursor stops sounds right, but it feels terrible: you pause to think and the video lurches in. What worked:

1. Edge-triggered settle. Zoom in once, when the cursor has been still (speed under 0.28 screen widths per second) for between 240 and 420 ms. Sitting still longer does not keep re-triggering.
2. Re-arm only after travel. Another settle zoom is only allowed after the cursor has moved more than 14% of the screen, or moved fast.
3. Hold, then release. Stay zoomed for 1650 ms, following the cursor with a small deadzone so tiny movements don't pan, then spring back to 1.0×.

// src/zoom_engine.rs
let fresh_settle = self.armed_after_travel
    && (240..=420).contains(&dwell_ms)
    && cursor.speed < 0.28
    && now_ms >= self.cooldown_until_ms;

Here's what it looks like in a recording:

And here it is running in the browser workbench, caught mid-zoom at 1.98× (left: full frame with the crop box, right: what ends up in the video):

You can also zoom by hand. In the studio, Record Combo… captures any mouse button, modifier+click, cursor shake, or key, and binds it to toggle lock, hold to zoom, or a 1.8× → 2.5× step. Here the camera is locked at 2.00× on an Export button:

Capture: the portal, a small C helper, and NVENC

Frames come from the standard ScreenCast portal (xdg-desktop-portal), which hands over a PipeWire stream. A standalone C helper (src/pw_portal_capture.c) negotiates DMA-BUF or MemFd buffers, maps each one, and pipes raw frames to FFmpeg, which encodes with NVENC (or libx264 if NVENC isn't available):

// src/pw_portal_capture.c
"ffmpeg -y -hide_banner -loglevel error "
"-f rawvideo -pix_fmt %s -video_size %ux%u -framerate %d -i pipe:0 "
"-c:v h264_nvenc -preset p4 -tune hq -rc vbr -cq 19 -b:v 0 -pix_fmt yuv420p -movflags +faststart \"%s\" "
"|| ffmpeg ... -c:v libx264 -preset veryfast -crf 19 ..."

Two details made this pleasant to use:

• The portal restore token is saved, so you pick a screen once instead of on every recording.
• The helper is a separate process, so a portal or PipeWire hiccup can't take the UI down with it.

To be clear about the cost: this path copies each frame once in system memory. It isn't zero-copy. Importing the DMA-BUF straight into the encoder would remove that copy, and that's a great contribution if you know VAAPI or NVENC surface import.

The UI: a recording island that isn't a window

The controls live in a small "island" at the top of the screen, built with GTK4 and gtk4-layer-shell. A layer-shell surface (zwlr_layer_shell_v1) is anchored to the output rather than managed like a window, so it doesn't steal focus or show up in the taskbar. If the compositor doesn't support layer-shell, it opens as a normal window instead.

Numbers

Measured during a 1080p60 recording on a hybrid AMD Radeon + NVIDIA RTX 3050 laptop, COSMIC, NVENC preset P4:

OBS and gpu-screen-recorder are excellent general-purpose recorders, and this isn't a replacement for them. HyperGSR does one thing they don't do out of the box: a camera that follows the cursor on Wayland. Your numbers will vary with GPU, compositor and resolution; gpu-recorder-rs --report prints the encoders and cursor planes it finds on your machine.

Building the landing page

I wanted the site to feel like an instrument, not a SaaS template, so I built it on a reference design system (Graphite from Refero) with strict rules: monochrome buttons, one accent color reserved for links, no emoji, a light/dark/system theme switch, and real code instead of illustrations. The full token set and the QA log are in DESIGN.md.

It works on phones too:

What I'd do differently

• Start from the cursor, not the encoder. I spent the first days on capture, but the cursor source decided the whole architecture. If you're building anything cursor-aware on Wayland, check drm_info for a cursor plane first.
• Tune the trigger before the spring. Almost every "this feels wrong" report came from when the camera zoomed, not how it moved.
• Measure the copy. "GPU recorder" invites the assumption of zero-copy. Profile your capture path and say what it actually does.

Try it, or help out

git clone https://github.com/KunalSiyag/gpu-recorder-rs
cd gpu-recorder-rs && ./install.sh
./target/release/gpu-recorder-rs

There's also a prebuilt x86_64 binary in the v1.0.0 release.

Each open issue names the file to start in:

• #1 Add Catppuccin Mocha, Tokyo Night and Nord themes to the island (good first issue, src/gui.rs)
• #2 Show pressed keys as an on-screen pill in the recorded MP4 (src/encoder.rs)
• #3 Add wallpaper padding and a drop-shadow frame to exports (good first issue, src/encoder.rs)

If you've solved the Wayland cursor problem a different way, I'd like to hear it in the comments. mathisdev7 took a different route in I'm 19 and I Built a Screen Studio Alternative for Linux with Rust and wgpu; it's worth reading next to this one.

Links: Landing page and live spring camera · Source on GitHub · Reverse-engineering notes