Agents & executor
The agent slot, what gets evaluated, and how it plugs in. ALE keeps this flexible by splitting the job in two: a deployer says what to do to run an agent; an executor decides where it runs. Pick a shape, write a small deployer, and you're in.
Generalist Computer Use Agents: CLI + GUI
Existing agent benchmarks tend to pick one surface. Terminal-Bench's Terminus drives a pure tmux terminal, with no GUI at all; OSWorld-style suites go the other way, handing the model only a native GUI action space: no terminal, no APIs or other tools. ALE deliberately needs both at once.
The reason is the work itself. A terminal is the most efficient interface for an LLM, and ever more software exposes one, so CLI control is the right default where it exists. But much real professional software either has no terminal at all, or exposes one whose commands and APIs cover far less than its GUI does, so the GUI is often the only complete way in. And many deliverables only count when they're produced through the application a professional actually uses (so a human can open, inspect, and edit the result) rather than synthesized wholesale by the model. Covering real workflows therefore takes an agent fluent in both: a generalist Computer Use Agent (CUA).
Two shapes of agent
A second axis is where the harness runs relative to the sandbox.
In-sandbox
The agent runs inside the sandbox. Many harnesses come with their own prompts, control loop, tools, and runtime, and are best left to run exactly as designed, so ALE injects the agent's CLI into the freshly-booted sandbox and launches it there, natively on the machine. Running in the OS it already has the shell and filesystem; ALE adds GUI access through the cua MCP bridge, so the same agent can also see the screen and click. It is the natural fit for harnesses that are already command-line tools, like Claude Code or Codex.
Out-of-sandbox
The agent's runtime stays outside the sandbox (in ALE's own process, or a container) and drives the machine remotely. Since none of it runs inside, it reaches in through two MCP bridges: a CLI bridge for the shell and files, and the GUI bridge. Its memory, sub-agents, and context management stay outside with it, never entering the sandbox. ALE-Claw is the in-tree reference.
The deployer: what to do
Whichever shape, an agent is added as a deployer, a small
Python class (BaseAgentDeployer) with three methods:
install(): stage the agent and apply its configuration: fetch or locate its CLI, wire in the model and tool knobs from the agent config, set up its MCP servers, check the API key.launch(prompt): run the agent to completion and return anAgentRunResult.parse_artifacts(): read the agent's native logs into a uniform trajectory.
The discipline that makes the split work: a deployer is plain Python that does all its I/O through a handle it's given, and never asks where it runs. Write it once; the executor decides the rest.
The executor: where it runs
The executor is that handle. It carries the work dir, the sandbox coordinates, the agent's config, and its environment, and runs the deployer on the substrate that matches the agent's shape:
| Agent shape | Executor | Runs the deployer… |
|---|---|---|
| In-sandbox | sandbox | Inside the sandbox itself. |
| Out-of-sandbox | local | In ALE's own Python process, on your host. |
docker | In a local container. |
docker walls the agent's runtime off from your host, so an agent
reaching for the filesystem touches the container, not your real working
files, safer than running it directly with local.
The MCP control surface
Two bridges make up the action space ALE exposes over the machine:
| Bridge | Action space |
|---|---|
| CLI vm_mcp_server | Run shell commands, read / write / list files, drive interactive PTY sessions: a thin set of system primitives, not opinionated tools. |
| GUI cua_mcp_server | Screenshot, click, type, scroll, drag, key-press: the desktop actions that turn a screen into an action space. |
Full per-tool parameters and return shapes: MCP tools.
An out-of-sandbox agent reaches the machine entirely through these two; an in-sandbox agent already has a native shell, so it takes only the GUI bridge. Either way, an agent can consume a bridge at two levels:
- As add-ons: mount the bridges' tools straight onto the agent's existing action space.
- As a substrate: build your own richer tools on top of them. ALE-Claw does this: its thick
read/write/edit/exec/computertools route through the bridges, with paging, truncation, and recovery layered on.
Why this stays flexible
Adding an agent is therefore small and composable: write a deployer, pick where it runs, and (if it's out-of-sandbox) decide how it consumes the two MCPs. The pieces mix freely. You could take an in-sandbox CLI like Claude Code, run it out of the sandbox, and hand it the two MCP bridges in place of a native shell. The deployer doesn't change, only the executor and transport do. That separation is what lets very different agents drop into one evaluation harness without special-casing any of them.