Screen

Screenshots, display geometry, and resizing the coordinate space.

Screenshot

const shot = await machine.screen.screenshot({ format: "png" });
shot.image; // base64
shot.display.width; // the geometry it was captured at

shot = machine.screen.screenshot(format="png")
shot["image"]              # base64
shot["display"]["width"]   # the geometry it was captured at

format is png, jpeg or webp, and quality applies to the lossy two. The image comes back base64 with the display it was taken at alongside it, so a result is self-describing and a coordinate can be checked against the frame that produced it.

Reads need no input lease.

Display

const display = await machine.screen.display();
display.width;
display.height;
display.up;

display = machine.screen.display()
display["width"], display["height"], display["up"]

up is what waitForDesktop polls. Before the desktop answers, this route raises compute_unavailable.

Resize

await machine.screen.resize(1920, 1080);

machine.screen.resize(1920, 1080)

Resize really resizes. The display server cannot do it in place, so the machine restarts the display and says so in the response. Reporting a new size while every subsequent screenshot came back at the old one would be worse than refusing.

That means a screenshot taken before a resize is stale for clicking. Take a fresh one:

await machine.screen.resize(1920, 1080);
const shot = await machine.screen.screenshot({ format: "png" });
await machine.mouse.click({ at: { x: 960, y: 540 } });

machine.screen.resize(1920, 1080)
shot = machine.screen.screenshot(format="png")
machine.mouse.click(at={"x": 960, "y": 540})

Resizing is an input action and takes the lease.

Live frames

For continuous frames rather than one at a time, see streams.