Self-Hosted Orbs

#!/usr/bin/env bash
# dispatch.sh
set -euo pipefail

# "$ORB_IMAGE" is the Amp orb image.

case "$ORB_OPERATION" in
	spawn)
		image="$ORB_IMAGE"
		docker image inspect "${ORB_SNAPSHOT_ID-}" >/dev/null 2>&1 && image="$ORB_SNAPSHOT_ID"
		docker start "$ORB_ID" 2>/dev/null ||
			docker run --init --platform linux/amd64 -d --name "$ORB_ID" \
				"$image" sleep infinity
		docker exec -d --env ORB_TOKEN "$ORB_ID" amp orb bootstrap
		;;
	pause)
		docker stop "$ORB_ID"
		[ -z "${ORB_SNAPSHOT_ID-}" ] || docker commit "$ORB_ID" "$ORB_SNAPSHOT_ID"
		;;
	delete)
		docker rm -f "$ORB_ID"
		;;
esac

Docker does not save memory, so processes in the orb stop when it pauses. The container keeps its files, so a waking orb runs .agents/resume, not .agents/setup. Project snapshots are local images, so they work only on this host.

# Needs the E2B CLI, `npm install e2b`, and E2B_API_KEY. Build the template once:
echo 'FROM us-central1-docker.pkg.dev/amp-prod-0/amp-orb/amp-orb:latest' \
    > e2b.Dockerfile
e2b template create amp-orbs --dockerfile e2b.Dockerfile \
    --cpu-count 4 --memory-mb 8192 --ready-cmd true
export E2B_TEMPLATE=amp-orbs

#!/usr/bin/env node
// dispatch.mjs
import { Sandbox } from 'e2b'

const {
	ORB_OPERATION, ORB_ID, ORB_TOKEN, AMP_URL, E2B_TEMPLATE,
} = process.env
const metadata = { orbId: ORB_ID }
const [existing] = await Sandbox.list({
	query: { metadata, state: ['running', 'paused'] },
}).nextItems()

if (ORB_OPERATION === 'spawn') {
	const timeoutMs = 60 * 60 * 1000
	const sandbox = existing
		? await Sandbox.connect(existing.sandboxId, { timeoutMs })
		: await Sandbox.create(E2B_TEMPLATE, {
			metadata, timeoutMs, lifecycle: { onTimeout: 'pause' },
		})
	const envs = { ORB_TOKEN }
	if (AMP_URL) envs.AMP_URL = AMP_URL
	const bootstrap = await sandbox.commands.run('amp orb bootstrap', {
		user: 'user', envs, background: true, timeoutMs: 0,
	})
	await bootstrap.disconnect()
} else if (ORB_OPERATION === 'pause' && existing) {
	await Sandbox.pause(existing.sandboxId)
} else if (ORB_OPERATION === 'delete' && existing) {
	await Sandbox.kill(existing.sandboxId)
}

E2B preserves the disk and memory on pause. This example also pauses on timeout after an hour rather than deleting the orb. It finds running and paused orbs by metadata, so the provider does not need to save sandbox IDs locally. It ignores ORB_SNAPSHOT_ID, so every new thread runs .agents/setup.

Amp supplies a Helm chart for GKE, EKS, and AKS.

It runs the orb provider and a dispatch program that creates orbs as pods and saves paused orbs to your storage bucket, so you do not write a dispatch program yourself. The bucket holds the orb’s files, not its memory, so processes in the orb stop when it pauses.

See Kubernetes to set it up.

#!/usr/bin/env bash
# dispatch.sh
set -euo pipefail
export PATH="/usr/local/bin:/opt/homebrew/bin:$PATH"

# "$ORB_IMAGE" is the Amp orb image.

ids=$(container list --all --quiet)
exists() { grep -Fxq "$ORB_ID" <<<"$ids"; }

case "$ORB_OPERATION" in
	spawn)
		image="$ORB_IMAGE"
		container image inspect "${ORB_SNAPSHOT_ID-}" >/dev/null 2>&1 && image="$ORB_SNAPSHOT_ID"
		if exists; then
			container start "$ORB_ID"
		else
			container run --detach --init --rosetta \
				--platform linux/amd64 --cpus 4 --memory 8G \
				--name "$ORB_ID" "$image" sleep infinity
		fi
		# A bare --env NAME copies the value from the host.
		container exec --detach --env ORB_TOKEN --env AMP_URL \
			"$ORB_ID" amp orb bootstrap
		;;
	pause)
		if exists; then
			container stop "$ORB_ID"
			[ -z "${ORB_SNAPSHOT_ID-}" ] || container commit "$ORB_ID" "$ORB_SNAPSHOT_ID"
		fi
		;;
	delete)
		if exists; then
			container delete --force "$ORB_ID"
		fi
		;;
	*)
		exit 1
		;;
esac

Apple containers do not save memory, so processes in the orb stop when it pauses. Saving project snapshots with container commit needs container 1.6.0 or later. Snapshots are local images, so they work only on this Mac.

#!/usr/bin/env python3
# dispatch.py
import os

import modal

# Needs `pip install modal` plus MODAL_TOKEN_ID and MODAL_TOKEN_SECRET.
APP = "amp-orbs"
app = modal.App.lookup(APP, create_if_missing=True)
snapshots = modal.Dict.from_name("amp-orb-snapshots", create_if_missing=True)
orb_id = os.environ["ORB_ID"]

try:
    sandbox = modal.Sandbox.from_name(APP, orb_id)
except modal.exception.NotFoundError:
    sandbox = None

match os.environ["ORB_OPERATION"]:
    case "spawn":
        if sandbox is None:
            # Modal has no pause, so resume from the last snapshot.
            snapshot = snapshots.get(orb_id)
            if snapshot:
                image = modal.Image.from_id(snapshot)
            else:
                image = modal.Image.from_registry(os.environ["ORB_IMAGE"])
            sandbox = modal.Sandbox.create(
                "sleep", "infinity", app=app, name=orb_id, image=image,
                cpu=4, memory=(16384, 16384), timeout=24 * 60 * 60,
            )
        # Modal execs run as root; point bootstrap at the image user's home.
        bootstrap = "setsid amp orb bootstrap </dev/null >/dev/null 2>&1 &"
        sandbox.exec(
            "bash", "-c", bootstrap,
            env={
                "ORB_TOKEN": os.environ["ORB_TOKEN"],
                "AMP_URL": os.environ.get("AMP_URL"),
                "HOME": "/home/user", "USER": "user",
            },
        ).wait()
    case "pause":
        if sandbox:
            image_id = sandbox.snapshot_filesystem(timeout=120).object_id
            snapshots[orb_id] = image_id
            sandbox.terminate()
    case "delete":
        if sandbox:
            sandbox.terminate()
        snapshots.pop(orb_id, None)

Modal’s filesystem snapshot keeps the orb’s files but not its memory, so processes in the orb stop when it pauses. This example ignores ORB_SNAPSHOT_ID, so every new thread runs .agents/setup.

#!/usr/bin/env node
// dispatch.mjs
import { APIError, Sandbox, Snapshot } from '@vercel/sandbox'

const { ORB_OPERATION, ORB_ID, ORB_IMAGE, ORB_TOKEN, AMP_URL } = process.env
const credentials = {
	token: process.env.VERCEL_TOKEN,
	teamId: process.env.VERCEL_TEAM_ID,
	projectId: process.env.VERCEL_PROJECT_ID,
}

if (ORB_OPERATION === 'spawn') {
	const sandbox = await Sandbox.getOrCreate({
		...credentials,
		name: ORB_ID,
		// Copy Amp's image to VCR, then set ORB_IMAGE to its VCR URL.
		image: ORB_IMAGE,
		resources: { vcpus: 4 },
		timeout: 60 * 60 * 1000,
		persistent: true,
		snapshotExpiration: 0,
		keepLastSnapshots: { count: 1, expiration: 0, deleteEvicted: true },
	})
	const setup = await sandbox.runCommand({
		cmd: '/bin/bash',
		args: ['-c', 'hostname localhost && mkdir -p /vercel/sandbox'],
		sudo: true,
	})
	if (setup.exitCode !== 0) throw new Error(await setup.stderr())
	await sandbox.runCommand({
		cmd: '/usr/bin/setpriv',
		args: [
			'--reuid=user', '--regid=user', '--init-groups', '--',
			'/usr/bin/env', `ORB_TOKEN=${ORB_TOKEN}`,
			...(AMP_URL ? [`AMP_URL=${AMP_URL}`] : []),
			'HOME=/home/user', 'USER=user', '/bin/bash', '-lc',
			'exec amp orb bootstrap',
		],
		sudo: true,
		detached: true,
	})
} else if (ORB_OPERATION === 'pause' || ORB_OPERATION === 'delete') {
	try {
		const sandbox = await Sandbox.get({
			...credentials, name: ORB_ID, resume: false,
		})
		if (ORB_OPERATION === 'delete') {
			if (sandbox.status === 'running') await sandbox.stop()
			for await (const saved of await sandbox.listSnapshots()) {
				if (saved.status !== 'created') continue
				const snapshot = await Snapshot.get({
					...credentials, snapshotId: saved.id,
				})
				await snapshot.delete()
			}
			await sandbox.delete()
		} else if (sandbox.status === 'running') await sandbox.stop()
	} catch (error) {
		if (!(error instanceof APIError && error.response.status === 404)) {
			throw error
		}
	}
}

Vercel saves the orb’s files when it stops, but not its memory, so processes in the orb stop when it pauses. This example ignores ORB_SNAPSHOT_ID, so every new thread runs .agents/setup.

#!/usr/bin/env python3
# dispatch.py
import os
from uuid import uuid4
from daytona import (
    Daytona, DaytonaNotFoundError, CreateSandboxFromImageParams,
    Resources, SessionExecuteRequest, SandboxState,
)

daytona = Daytona()
operation = os.environ['ORB_OPERATION']
try:
    sandbox = daytona.get(os.environ['ORB_ID'])
except DaytonaNotFoundError:
    sandbox = None

if operation == 'spawn':
    if sandbox is None:
        sandbox = daytona.create(CreateSandboxFromImageParams(
            name=os.environ['ORB_ID'], image=os.environ['ORB_IMAGE'],
            os_user='user', resources=Resources(cpu=4, memory=8, disk=10),
            auto_stop_interval=0, auto_delete_interval=-1,
        ), timeout=240)
    elif sandbox.state != SandboxState.STARTED:
        sandbox.start()
    env = {key: os.environ[key] for key in ('ORB_TOKEN', 'AMP_URL')
           if key in os.environ}
    sandbox.update_env(env, unset=['AMP_URL'])
    session = str(uuid4())
    sandbox.process.create_session(session)
    sandbox.process.execute_session_command(session, SessionExecuteRequest(
        command="bash -lc 'exec amp orb bootstrap'", run_async=True,
    ))
elif operation == 'pause' and sandbox:
    if sandbox.state == SandboxState.STARTED:
        sandbox.stop()
elif operation == 'delete' and sandbox:
    sandbox.delete()

Daytona keeps the orb’s files when it stops, but not its memory, so processes in the orb stop when it pauses. This example ignores ORB_SNAPSHOT_ID, so every new thread runs .agents/setup.

#!/usr/bin/env bash
# dispatch.sh
set -euo pipefail

# Pause/resume currently requires access enabled by exe.dev support.
status=$(ssh exe.dev ls "$ORB_ID" --json |
    jq -r --arg id "$ORB_ID" '.vms[] | select(.vm_name == $id) | .status')

case "$ORB_OPERATION" in
    spawn)
        if [[ -z "$status" ]]; then
            ssh exe.dev new --name="$ORB_ID" --image="$ORB_IMAGE" \
                --cpu=2 --memory=8G --no-email
        elif [[ "$status" == paused ]]; then
            ssh exe.dev resume "$ORB_ID"
        fi
        {
            printf 'export ORB_TOKEN=%q\n' "$ORB_TOKEN"
            if [[ -n "${AMP_URL:-}" ]]; then
                printf 'export AMP_URL=%q\n' "$AMP_URL"
            fi
            cat <<'BOOTSTRAP'
sudo -u user --preserve-env=ORB_TOKEN,AMP_URL bash -lc \
    'cd "$HOME"; nohup amp orb bootstrap </dev/null >.amp-bootstrap.log 2>&1 &'
BOOTSTRAP
        } | ssh "$ORB_ID.exe.xyz" bash -s
        ;;
    pause)
        if [[ "$status" == running ]]; then
            ssh exe.dev pause "$ORB_ID"
        fi
        ;;
    delete)
        if [[ -n "$status" ]]; then
            ssh exe.dev rm "$ORB_ID"
        fi
        ;;
esac

exe.dev saves memory on pause, so processes keep running when the orb wakes. This example ignores ORB_SNAPSHOT_ID, so every new thread runs .agents/setup.

Install the Boat CLI and sign in with boat login. The controller needs Bash, jq, and OpenSSH. Enable snapshots in Boat’s settings and keep zero data retention off.

#!/usr/bin/env bash
# dispatch.sh
set -euo pipefail
export BOAT_NO_UPDATE=1
# Keep this directory on persistent storage.
STATE="${BOAT_ORB_STATE:-$HOME/.amp-boat-orbs}"
mkdir -p "$STATE"
trap 'echo "Boat $ORB_OPERATION $ORB_ID failed at line $LINENO" >&2' ERR
MAP="$STATE/$ORB_ID"
id=""
state=""
if [[ -s "$MAP" ]]; then
    id=$(cat "$MAP")
    state=$(boat info "$id" --json | jq -er '.sandbox.state')
fi
case "$ORB_OPERATION" in
    spawn)
        if [[ -z "$id" ]]; then
            # Read all JSON Lines before parsing. Increase the TTL for longer turns.
            out=$(boat new --json --ttl 3600 --no-env)
            printf '%s\n' "$out" | jq -ser 'map(select(.event == "ready"))[0].id // empty' > "$MAP"
            id=$(cat "$MAP")
        else
            while [[ "$state" == stopping ]]; do
                sleep 2
                state=$(boat info "$id" --json | jq -er '.sandbox.state')
            done
            if [[ "$state" == stopped || "$state" == error ]]; then
                boat resume "$id" >/dev/null
            fi
        fi
        {
            printf 'export ORB_TOKEN=%q AMP_URL=%q\n' "$ORB_TOKEN" "${AMP_URL:-https://ampcode.com}"
            printf '%s\n' \
                'export PATH="$HOME/.local/bin:$HOME/.amp/bin:$PATH"' \
                'command -v amp >/dev/null || curl -fsSL https://ampcode.com/install.sh | bash' \
                'nohup amp orb bootstrap </dev/null >~/.amp-bootstrap.log 2>&1 &'
        } | boat ssh "$id" bash -se -o pipefail
        ;;
    pause)
        if [[ -n "$id" && "$state" != stopped ]]; then
            [[ "$state" == stopping ]] || boat stop "$id" >/dev/null
            until [[ "$(boat info "$id" --json | jq -er '.sandbox.state')" == stopped ]]; do
                sleep 2
            done
        fi
        ;;
    delete)
        if [[ -n "$id" ]]; then
            boat delete "$id" --yes
            rm -f "$MAP"
        fi
        ;;
esac

Boat keeps home files for 14 days after a pause, but not /tmp or ~/.cache, and memory may not survive. This example uses Boat’s base machine instead of ORB_IMAGE and ignores ORB_SNAPSHOT_ID, so every new thread runs .agents/setup.