---
title: The desktop
description: Screenshots, pointer, keyboard, clipboard, and the display geometry every coordinate is in.
---

The desktop is an ordinary Linux desktop, driven the way a person would drive it. The image
ships an `agent` user that owns the session, and a `root` user.

## Coordinates

Every coordinate is in the pixel space of the machine's **current** display. A coordinate
outside it is **refused, not clamped**: a click 200 pixels off the edge is a bug worth surfacing,
not one to land somewhere plausible.

Nothing is scaled for you. Take a screenshot, read the coordinates off it, click them.

```ts
const shot = await machine.screen.screenshot({ format: "png" });
shot.display.width; // the geometry this image was captured at
```

Resizing changes that space, so a screenshot taken before a resize is stale for clicking.

## Screen

| Call                                 | What it does                                   |
| ------------------------------------ | ---------------------------------------------- |
| `GET /machines/{id}/screenshot`      | capture the screen; `png`, `jpeg` or `webp`    |
| `GET /machines/{id}/display`         | width, height, and whether the desktop is `up` |
| `POST /machines/{id}/display/resize` | change the display geometry                    |

A screenshot comes back base64 in `image`, with the `display` it was taken at alongside it, so a
result is self-describing and a coordinate can be checked against the frame that produced it.
`quality` applies to the lossy formats.

For continuous frames rather than one at a time, use
[the screen channel](/docs/reference/realtime).

## Pointer

| Call                               | Notes                                                 |
| ---------------------------------- | ----------------------------------------------------- |
| `POST /machines/{id}/mouse/move`   | absolute `x`, `y`                                     |
| `POST /machines/{id}/mouse/click`  | `at` optional — omit it to click where the pointer is |
| `POST /machines/{id}/mouse/drag`   | `from` optional, `to` required, `steps` for the path  |
| `POST /machines/{id}/mouse/scroll` | `direction`, `amount` (default 3), optional `at`      |

Buttons are `left`, `middle` and `right`. `count: 2` is a double click; the SDKs spell that
`doubleClick`. `down` and `up` let you hold a button across several calls.

There is no read-only pointer query. Moving the pointer to find out where it is defeats the
question, so the Anthropic adapter withholds `cursor_position` rather than declaring a tool that
cannot work.

## Keyboard

| Call                                  | Notes                                          |
| ------------------------------------- | ---------------------------------------------- |
| `POST /machines/{id}/keyboard/type`   | a string, with an optional per-character delay |
| `POST /machines/{id}/keyboard/key`    | one key by name, e.g. `Return`, `Tab`, `F6`    |
| `POST /machines/{id}/keyboard/hotkey` | a combination, e.g. `ctrl` + `l`               |

`type` is for text and `key` is for keys. Typing `"Return"` types seven characters.

## Clipboard

```ts
await machine.clipboard.write("hello");
const { text, selection } = await machine.clipboard.read();
```

X11 has two selections and both are addressable: `clipboard` (the default, what Ctrl-V pastes)
and `primary` (what a middle click pastes). Reading is a read, so it needs no input lease.

## The input lease

Anything that types or clicks needs the machine's single input lease. Reads — screenshots,
display geometry, clipboard reads — never do. Both SDKs acquire and renew the lease for you on
the first call that needs it, and never re-acquire one they lost.

That last part is the point: a lost lease means a person took over, and silently taking it back
would put an agent and a person on the same keyboard. See
[sessions and input control](/docs/reference/sessions).