katie computer helper

a small wayland sidecar that owns an xdg desktop portal remote desktop session

it reads newline delimited json requests from stdin and writes one json response per line to stdout

the helper deliberately has no network support and no shell action

requirements

• wayland compositor with remote desktop portal support
• pipewire
• libei 1.0 or newer
• rust 1.97 or newer

on first launch the desktop portal asks which screen to share and whether keyboard and pointer control are allowed

build

cargo build --release

protocol

every request needs a unique string id

{"id":"1","action":"status"}
{"id":"2","action":"observe","path":"/tmp/katie-screen.webp"}
{"id":"3","action":"move","x":0.5,"y":0.5}
{"id":"4","action":"click","x":0.5,"y":0.5,"button":"left"}
{"id":"5","action":"button","button":"left","pressed":true}
{"id":"6","action":"button","button":"left","pressed":false}
{"id":"7","action":"drag","fromX":0.2,"fromY":0.3,"toX":0.7,"toY":0.8,"button":"left"}
{"id":"8","action":"scroll","deltaY":120}
{"id":"9","action":"keyTap","code":28}
{"id":"10","action":"key","code":29,"pressed":true}
{"id":"11","action":"type","text":"hello from katie"}
{"id":"12","action":"key","code":29,"pressed":false}
{"id":"13","action":"stop"}

coordinates are normalized from 0 to 1 over the selected portal stream

keyboard codes and pointer buttons use linux evdev codes rather than x11 key symbols

successful responses look like

{"id":"4","ok":true,"result":{"x":960.0,"y":540.0,"button":"left"}}

errors are request scoped and do not stop the helper

{"id":"4","ok":false,"error":"absolute pointer device is not ready"}

the first output after portal setup is a startup response containing selected stream metadata and an optional replacement restore token

{"id":"startup","ok":true,"result":{"ready":true,"nodeId":57,"mappingId":"DP-1","logicalSize":[1920,1080],"restoreToken":"..."}}

save the latest restore token securely and provide it on the next launch as KATIE_COMPUTER_RESTORE_TOKEN

portal restore tokens are single use and must be replaced after every successful start

safety boundary

the parent application should still enforce authentication confirmation policy action limits and auditing

the helper itself only supports capture pointer keyboard and direct text actions

it rejects relative screenshot paths out of range coordinates oversized requests unsupported image formats and invalid key codes