Tools

The model-agnostic toolkit — eight JSON-schema tools and a dispatcher, with no vendor package in it.

machine.tools.generic() gives any agent loop a toolkit that works with any model. The definitions are plain JSON Schema and the executors are ordinary functions, so no provider SDK needs to be installed.

const toolkit = machine.tools.generic();

toolkit.definitions; // [{ name, description, input_schema }]

const result = await toolkit.execute("computer_screenshot", {});
result.content; // [{ type: "image", data, media_type }] or [{ type: "text", text }]
result.is_error;

toolkit = machine.tools.generic()

toolkit["definitions"]   # [{ "name", "description", "input_schema" }]

result = toolkit["execute"]("computer_screenshot", {})
result["content"]        # [{"type": "image", "data": ..., "media_type": ...}]
result["is_error"]

The eight tools

Name	What it does
`computer_screenshot`	capture the screen; returns the image and its geometry
`computer_click`	click at a point, with a button and a count
`computer_type`	type text
`computer_key`	press a key or a combination
`computer_scroll`	scroll, optionally after moving the pointer
`terminal_exec`	run a command as argv
`file_read`	read a file
`file_write`	write a file

The names, argument shapes and result shapes are a published contract: a field may be added, a meaning may not change.

TOOL_DEFINITIONS and TOOL_NAMES are importable at the top level in both languages, for building a request before you have a machine:

import { TOOL_DEFINITIONS, TOOL_NAMES } from "@raster/sdk";

from raster import TOOL_DEFINITIONS, TOOL_NAMES

Failures are results, not exceptions

execute does not throw on a refused action. It returns is_error: true with content saying why:

const result = await toolkit.execute("computer_click", { at: { x: 99999, y: 0 } });
result.is_error; // true
result.content[0]; // { type: "text", text: "invalid_request: ..." }

result = toolkit["execute"]("computer_click", {"at": {"x": 99999, "y": 0}})
result["is_error"]                # True
result["content"][0]              # {"type": "text", "text": "invalid_request: ..."}

That is the right shape for a loop. A model told the click landed outside the display can take another screenshot; a model handed an exception through a broken harness learns nothing.

Using it in a loop

const toolkit = machine.tools.generic();

// hand the definitions to whatever model you use
const tools = toolkit.definitions.map((d) => ({
  name: d.name,
  description: d.description,
  input_schema: d.input_schema,
}));

// then dispatch whatever comes back
for (const call of modelToolCalls) {
  const result = await toolkit.execute(call.name, call.input);
  send(call.id, result);
}

toolkit = machine.tools.generic()

tools = [
    {"name": d["name"], "description": d["description"], "input_schema": d["input_schema"]}
    for d in toolkit["definitions"]
]

for call in model_tool_calls:
    result = toolkit["execute"](call["name"], call["input"])
    send(call["id"], result)

Mixing with a vendor adapter

The integrations cover the GUI in a vendor’s own tool vocabulary. Mixing in one canonical tool — usually terminal_exec — gives the model a shell without a terminal window, and both reach the same machine:

const exec = toolkit.definitions.find((d) => d.name === "terminal_exec")!;

const tools = [
  adapter.tool, // the vendor's computer toolset
  { name: exec.name, description: exec.description, input_schema: exec.input_schema },
];

Passing the canonical definition through as-is, rather than restating it, is what keeps the two from drifting. Every example in integrations does exactly this.