Set up AWS

One-time setup to run ALE on AWS: sign in, create a network, and copy the sandbox images into your account. Then point an experiment at the AWS environment and run. Credentials live in your AWS CLI profile; everything else is created once below.

Sign in

Create an AWS account if you don't have one, install the AWS CLI, and sign in. In the console make an IAM user with AdministratorAccess, create an access key for it, then:

aws configure                 # paste the access key, region us-east-1, output json
export AWS_REGION=us-east-1
export ACCOUNT=$(aws sts get-caller-identity --query Account --output text)

① Create the network

An ale-vpc network with a public subnet in three Availability Zones (tried in order if one is short on capacity), and a security group ale-sandbox opening the port ALE needs (5000; 3389 to view a Windows desktop).

VPC=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
  --tag-specifications 'ResourceType=vpc,Tags=[{Key=project,Value=ale},{Key=Name,Value=ale-vpc}]' \
  --query Vpc.VpcId --output text)
aws ec2 modify-vpc-attribute --vpc-id $VPC --enable-dns-hostnames

IGW=$(aws ec2 create-internet-gateway --query InternetGateway.InternetGatewayId --output text)
aws ec2 attach-internet-gateway --internet-gateway-id $IGW --vpc-id $VPC
RT=$(aws ec2 describe-route-tables --filters "Name=vpc-id,Values=$VPC" \
  "Name=association.main,Values=true" --query 'RouteTables[0].RouteTableId' --output text)
aws ec2 create-route --route-table-id $RT --destination-cidr-block 0.0.0.0/0 --gateway-id $IGW

i=1; for az in us-east-1a us-east-1b us-east-1c; do
  SN=$(aws ec2 create-subnet --vpc-id $VPC --availability-zone $az --cidr-block 10.0.$i.0/24 \
    --tag-specifications 'ResourceType=subnet,Tags=[{Key=project,Value=ale}]' \
    --query Subnet.SubnetId --output text)
  aws ec2 modify-subnet-attribute --subnet-id $SN --map-public-ip-on-launch
  aws ec2 associate-route-table --route-table-id $RT --subnet-id $SN; i=$((i+1))
done

SG=$(aws ec2 create-security-group --group-name ale-sandbox \
  --description "ALE sandbox ingress" --vpc-id $VPC --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id $SG --protocol tcp --port 5000 --cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-id $SG --protocol tcp --port 3389 --cidr 0.0.0.0/0

② Copy the sandbox images

A task boots from a prebaked image (the OS, the professional software, and the task data, ready to go). ALE publishes the following images publicly from account 435756742641 in us-east-1. Copy the one(s) you need into your account once.

The AMIs have public launch permission and their backing EBS snapshots have public create-volume permission. A cross-account copy creates an independent AMI and snapshot owned by your account. The copies are private by default.

# copy an image into your account (repeat per image you need)
fam=ale-ubuntu22                       # or ale-win10 / ale-win-server
src=$(aws ec2 describe-images --region us-east-1 --owners 435756742641 \
  --filters "Name=description,Values=$fam" "Name=state,Values=available" \
  --query 'sort_by(Images,&CreationDate)[-1].ImageId' --output text)
test "$src" != None
new=$(aws ec2 copy-image --region us-east-1 --source-region us-east-1 --source-image-id $src \
  --name $fam --description $fam --query ImageId --output text)
aws ec2 create-tags --region us-east-1 --resources $new \
  --tags Key=Name,Value=$fam Key=ale:image-family,Value=$fam
aws ec2 wait image-available --region us-east-1 --image-ids $new

The environment config refers to images by these family names, so once copied they're picked up automatically. AMI tags are not copied across accounts, which is why the command adds both tags explicitly. Copying a Windows image can take a while because it has a large backing disk.

③ Keep run output in S3 (recommended for real runs)

With output_path: local each run's output is fetched back to your machine over the sandbox's control port — fine for a quick demo, but for large outputs at scale that's slow and flaky and drags down overall throughput. So, exactly as on Google Cloud (where output goes to a GCS bucket), the recommended path is to have the sandbox upload straight to S3. Output is organized by run id, so you can find each run's files afterward. Create the bucket and a role that lets the sandbox upload to it:

aws s3api create-bucket --bucket ale-run-results-$ACCOUNT --region us-east-1

aws iam create-role --role-name ale-sandbox --assume-role-policy-document '{"Version":"2012-10-17",
  "Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}'
aws iam put-role-policy --role-name ale-sandbox --policy-name s3 --policy-document '{"Version":"2012-10-17",
  "Statement":[{"Effect":"Allow","Action":["s3:PutObject","s3:ListBucket"],
    "Resource":["arn:aws:s3:::ale-run-results-'$ACCOUNT'","arn:aws:s3:::ale-run-results-'$ACCOUNT'/*"]}]}'
aws iam create-instance-profile --instance-profile-name ale-sandbox
aws iam add-role-to-instance-profile --instance-profile-name ale-sandbox --role-name ale-sandbox

The snapshots already reference the ale-sandbox instance profile, so once it exists the sandbox is launched with it attached (that's what lets the in-box aws CLI write to the bucket). All you do to switch on S3 output is set output_path: s3://ale-run-results-<account> in configs/environments/environment_aws.yaml. (If the profile doesn't exist it's skipped for local runs, so you only need it for S3 output — and if output_path is a bucket but the profile is missing, the run fails fast with a clear message.)

④ Run

Point your experiment at the AWS environment (environment: configs/environments/environment_aws.yaml) and run:

uv run python -m ale_run run my_experiment.yaml