---
title: The realtime gateway
description: Live screen, input, terminal and event channels over one websocket protocol.
---

Frames, input and pty bytes do not go through the API — they go through a websocket gateway the
API mints a ticket for. The dashboard's desktop, terminal and file views run on it, and so do
`machine.screen.stream()` and `machine.terminal.connect()`.

## Getting a ticket

```
POST /v1/machines/{machine_id}/connect
  ->  { token, gateway_url, expires_at, capabilities }
```

The ticket is short-lived and **carries no credential of yours**. The gateway exchanges it
server-side for the machine's actual connection details, so the machine's address and its guest
bearer token never cross into a browser.

`capabilities` is the subset of `screen:read`, `input:write` and `terminal:write` this ticket
was granted. A ticket is scoped to one machine and one session.

## Connecting

```
wss://gateway.example/machines/{machine_id}/screen
wss://gateway.example/machines/{machine_id}/input
wss://gateway.example/machines/{machine_id}/terminal/{terminal_id}
wss://gateway.example/machines/{machine_id}/events
```

The token is presented as the websocket **subprotocol**:

```
Sec-WebSocket-Protocol: computer.v1, ticket.<token>
```

Not a query parameter: query strings land in access logs, proxy logs and browser history, and
this token is a credential. Browsers cannot set headers on a websocket handshake, so the
subprotocol field is the only place left.

## Close codes

The gateway accepts the handshake and immediately closes it, rather than refusing at the HTTP
layer — a browser cannot read the status of a rejected websocket handshake, so refusing there
would leave a client with "it failed" and nothing else.

| Code   | Meaning                                                           |
| ------ | ----------------------------------------------------------------- |
| `4401` | ticket missing, malformed, or expired                             |
| `4403` | ticket does not carry the capability this channel needs           |
| `4404` | no such machine in the ticket's organization, or no such terminal |
| `4409` | machine is not running, or the terminal already has a client      |
| `4429` | too many sockets for this machine or organization                 |
| `4503` | the control plane could not be reached; retry                     |

`4503` is listed separately because it is **not** a client error. A client that treats an
unreachable control plane as a bad ticket will mint a new one and fail the same way forever.

## Framing

One rule across every channel, so a client needs one dispatcher:

- **Text frames are JSON control messages**, always an object with a `type`.
- **Binary frames are payload** — pixels or pty bytes, never JSON.

Unknown `type` values are ignored rather than fatal, so a newer gateway can add a message
without breaking an older viewer.

## The channels

### screen

Server to client, capability `screen:read`. The gateway consumes the guest's multipart MJPEG
stream and republishes each part as one binary frame, so the browser never parses multipart.
Each frame is a 20-byte little-endian header — magic `CSF1`, sequence, geometry, timestamp —
followed by the image.

### input

Client to server, capability `input:write`. All control messages, no binary frames:
`mouse.move`, `mouse.click`, `mouse.drag`, `mouse.scroll`, `key.type`, `key.press`,
`key.hotkey`, `screen.resize`.

Every message carries the lease `fence`. Coordinates are **guest pixels, not CSS pixels** — the
viewer owns the mapping across scaling and letterboxing, because only the viewer knows how it is
drawing, and a gateway that guessed would be wrong for every client that draws differently.

Renewal is not on this socket. Renew through the API and, if the fence changed, reconnect the
input socket with a fresh ticket. See [input control](/docs/reference/sessions).

### terminal

Bidirectional, capability `terminal:write`. Bytes go both ways unmodified. A terminal that
already has a client refuses a second at the handshake with `4409`.

### events

Server to client. The machine's [event log](/docs/reference/machines#the-event-log) as it
happens, rather than polled.

## Through the SDKs

```ts
const stream = await machine.screen.stream();
for await (const frame of stream) render(frame.image);
stream.close();
```

```python
with machine.screen.stream() as stream:
    for frame in stream:
        render(frame.image)
```

Both mint the ticket, open the socket, decode the header and hand you frames. Screen streams
open a session but never take the input lease, so watching a desktop cannot evict whoever is
driving it.

In Python this needs the `realtime` extra: `pip install "raster[realtime]"`. Most callers
are agent loops that screenshot and click over ordinary HTTP and never open a socket, so the
websocket dependency is opt-in rather than paid for by everybody.