Set up AWS

One-time setup to run ALE on AWS: sign in, create a network, and copy the sandbox images into your account. Then point an experiment at the AWS environment and run. Credentials live in your AWS CLI profile; everything else is created once below.

Sign in

Create an AWS account if you don't have one, install the AWS CLI, and sign in. In the console make an IAM user with AdministratorAccess, create an access key for it, then:

aws configure                 # paste the access key, region us-east-1, output json
export AWS_REGION=us-east-1
export ACCOUNT=$(aws sts get-caller-identity --query Account --output text)

① Create the network

An ale-vpc network with a public subnet in three Availability Zones (tried in order if one is short on capacity), and a security group ale-sandbox opening the port ALE needs (5000; 3389 to view a Windows desktop).

VPC=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
  --tag-specifications 'ResourceType=vpc,Tags=[{Key=project,Value=ale},{Key=Name,Value=ale-vpc}]' \
  --query Vpc.VpcId --output text)
aws ec2 modify-vpc-attribute --vpc-id $VPC --enable-dns-hostnames

IGW=$(aws ec2 create-internet-gateway --query InternetGateway.InternetGatewayId --output text)
aws ec2 attach-internet-gateway --internet-gateway-id $IGW --vpc-id $VPC
RT=$(aws ec2 describe-route-tables --filters "Name=vpc-id,Values=$VPC" \
  "Name=association.main,Values=true" --query 'RouteTables[0].RouteTableId' --output text)
aws ec2 create-route --route-table-id $RT --destination-cidr-block 0.0.0.0/0 --gateway-id $IGW

i=1; for az in us-east-1a us-east-1b us-east-1c; do
  SN=$(aws ec2 create-subnet --vpc-id $VPC --availability-zone $az --cidr-block 10.0.$i.0/24 \
    --tag-specifications 'ResourceType=subnet,Tags=[{Key=project,Value=ale}]' \
    --query Subnet.SubnetId --output text)
  aws ec2 modify-subnet-attribute --subnet-id $SN --map-public-ip-on-launch
  aws ec2 associate-route-table --route-table-id $RT --subnet-id $SN; i=$((i+1))
done

SG=$(aws ec2 create-security-group --group-name ale-sandbox \
  --description "ALE sandbox ingress" --vpc-id $VPC --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id $SG --protocol tcp --port 5000 --cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-id $SG --protocol tcp --port 3389 --cidr 0.0.0.0/0

② Copy the sandbox images

A task boots from a prebaked image (the OS, the professional software, and the task data, ready to go). ALE publishes them publicly; copy the one(s) you need into your account once. Pick by what your tasks run on:

• ale-ubuntu22 — Linux (the bulk of tasks)
• ale-win10 or ale-win-server — Windows; either one works, pick one

# copy an image into your account (repeat per image you need)
fam=ale-ubuntu22                       # or ale-win10 / ale-win-server
src=$(aws ec2 describe-images --region us-east-1 --owners 670060057810 \
  --filters "Name=description,Values=$fam" --query 'Images[0].ImageId' --output text)
new=$(aws ec2 copy-image --source-region us-east-1 --source-image-id $src \
  --name $fam --description $fam --query ImageId --output text)
aws ec2 create-tags --resources $new --tags Key=ale:image-family,Value=$fam

The environment config refers to images by these family names, so once copied they're picked up automatically. (Copying a Windows image takes a while — it's a large disk.)

③ Keep run output in S3 (recommended for real runs)

With output_path: local each run's output is fetched back to your machine over the sandbox's control port — fine for a quick demo, but for large outputs at scale that's slow and flaky and drags down overall throughput. So, exactly as on Google Cloud (where output goes to a GCS bucket), the recommended path is to have the sandbox upload straight to S3. Output is organized by run id, so you can find each run's files afterward. Create the bucket and a role that lets the sandbox upload to it:

aws s3api create-bucket --bucket ale-run-results-$ACCOUNT --region us-east-1

aws iam create-role --role-name ale-sandbox --assume-role-policy-document '{"Version":"2012-10-17",
  "Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}'
aws iam put-role-policy --role-name ale-sandbox --policy-name s3 --policy-document '{"Version":"2012-10-17",
  "Statement":[{"Effect":"Allow","Action":["s3:PutObject","s3:ListBucket"],
    "Resource":["arn:aws:s3:::ale-run-results-'$ACCOUNT'","arn:aws:s3:::ale-run-results-'$ACCOUNT'/*"]}]}'
aws iam create-instance-profile --instance-profile-name ale-sandbox
aws iam add-role-to-instance-profile --instance-profile-name ale-sandbox --role-name ale-sandbox

The snapshots already reference the ale-sandbox instance profile, so once it exists the sandbox is launched with it attached (that's what lets the in-box aws CLI write to the bucket). All you do to switch on S3 output is set output_path: s3://ale-run-results-<account> in configs/environments/environment_aws.yaml. (If the profile doesn't exist it's skipped for local runs, so you only need it for S3 output — and if output_path is a bucket but the profile is missing, the run fails fast with a clear message.)

④ Run

Point your experiment at the AWS environment (environment: configs/environments/environment_aws.yaml) and run:

uv run python -m ale_run run my_experiment.yaml