---
title: OpenAI
description: Two surfaces over one translation — the Responses API's computer tool, and the Agents SDK's Computer.
---

`@raster/openai` ships two adapters over the same nine-action translation: `computerUse` for
the Responses API's `computer_use_preview` tool, and `agentComputer` for the Agents SDK's
`computerTool`. Neither owns a conversation, and neither is imported by the canonical SDK.

```bash
npm install @raster/openai openai
```

## The Responses API

```ts
import OpenAI from "openai";
import { Client } from "@raster/sdk";
import { computerUse } from "@raster/openai";

const machine = await new Client().machines.create({ size: "standard" });
const display = await machine.waitForDesktop();

// the display the adapter declares is the display it just measured, so the
// coordinate space the model reasons in cannot drift from the real one.
const adapter = await computerUse(machine, { display });
```

```ts
const model = new OpenAI();
let input = [{ role: "user", content: task }];
let previousResponseId;

for (let turn = 0; turn < 24; turn++) {
  const response = await model.responses.create({
    model: "computer-use-preview",
    tools: [adapter.tool],
    input,
    truncation: "auto", // computer use needs this: a screenshot per turn
    ...(previousResponseId ? { previous_response_id: previousResponseId } : {}),
  });

  previousResponseId = response.id;
  const next = await adapter.handleAll(response);
  if (next.length === 0) break;
  input = next;
}
```

`adapter.tool` is sized to the machine's real display. `handle` returns the input **items** to
send back — the `computer_call_output` carrying a fresh screenshot, plus, when something went
wrong, one user message saying what. The wire format has no error field on a computer output, so
a failure that is not said out loud is a model staring at an unchanged screen with no idea why.

### Safety checks

```ts
const adapter = await computerUse(machine, {
  display,
  onSafetyCheck: (call) => askAHuman(call),
});
```

**There is no default yes.** A pending safety check is the API asking a human-owned question, and
a library that answers it for everyone has removed the only point of asking. Calls whose checks
are not acknowledged are answered without being performed.

Other options: `environment` (what the model is told it is driving; the image is `ubuntu`),
`waitSeconds` (the protocol carries no duration for a `wait` action), `scrollPixelsPerClick`, and
`sleep`.

## The Agents SDK

```ts
import { Agent, computerTool, run } from "@openai/agents";
import { agentComputer } from "@raster/openai";

const agent = new Agent({
  name: "operator",
  model: "computer-use-preview",
  instructions: "You drive a real Linux desktop. Take a screenshot whenever you need to look.",
  tools: [
    computerTool({
      computer: await agentComputer(machine, { dimensions: [display.width, display.height] }),
    }),
  ],
  modelSettings: { truncation: "auto" },
});

const result = await run(agent, "Open Chromium, load https://example.com, and read the heading.");
```

`agentComputer` returns a machine as the Agents SDK's `Computer` interface. The SDK runs the loop
itself, so unlike the Responses adapter there is nothing to call per turn: it asks for actions
and this performs them. Same nine actions, same translation, different door.

The package depends only on `@openai/agents-core` types — install `@openai/agents` yourself to
run it.

Options are the same minus the safety-check hook, which the Agents SDK handles itself:
`dimensions` (skips the display lookup when you already know the geometry), `environment`,
`waitSeconds`, `scrollPixelsPerClick`, `sleep`.

## Adding a shell

The computer tool covers the GUI. One canonical tool from the SDK gives the model a shell:

```ts
const generic = machine.tools.generic();
const exec = generic.definitions.find((d) => d.name === "terminal_exec")!;

const tools = [
  adapter.tool,
  {
    type: "function",
    name: exec.name,
    description: exec.description,
    parameters: exec.input_schema,
    strict: false,
  },
];
```

The adapter does not touch your own function calls. Handle them as usual and push the outputs
alongside what `handleAll` returned:

```ts
const next = await adapter.handleAll(response);

for (const item of response.output) {
  if (item.type !== "function_call") continue;
  const result = await generic.execute(item.name, JSON.parse(item.arguments));
  next.push({
    type: "function_call_output",
    call_id: item.call_id,
    output: result.content.map((p) => (p.type === "text" ? p.text : "[image]")).join("\n"),
  });
}
```

## Also exported

`perform`, `describe` and `DEFAULT_RUNTIME` are the shared translation the two adapters sit on,
and `keyName` / `keyNames` map OpenAI's key vocabulary onto the guest's. Reach for them if you are
building a third surface rather than using one of these two.

Working examples live at `packages/openai/examples/computer-use.ts` and
`packages/openai/examples/agents-sdk.ts`.