Sandboxes for RL rollouts at scale
Run your whole RL loop on one platform, GPUs included.
RL compute environments
Deterministic resets, parallel rollouts, long-horizon state, and GPUs on-demand.
Easily run RL rollouts
Run Docker-in-Docker
Run any existing Docker image in a sandbox, giving you a fully reproducible environment between your local and the cloud.
Fan out sandboxes
Spin up thousands of sandboxes, and write outputs to a shared volume.
Attach GPUs
Add a GPU to any sandbox with just a single line of code.
# Run the policy update on a GPU — same SDK
from beam import function, Volume
@function(
gpu="H100",
volumes=[Volume(name="trajectories", mount_path="./data")],
)
def update_policy():
batch = load_trajectories("./data")
loss = policy.train_step(batch)
save_checkpoint("./data/policy-v43.pt")
# Pause a stateful environment and pick it back up later
snapshot_id = env.snapshot_memory()
env2 = Sandbox().create_from_memory_snapshot(snapshot_id)Frequently asked questions
How fast can a snapshot restore?
A restore is just booting a new sandbox from your saved snapshot, which takes under a second with dependencies included. Every episode starts from the identical filesystem state, and nothing from the previous episode leaks in.
Can I pause and resume an episode mid-run?
Yes. Memory snapshots capture the sandbox's full state — running processes and exposed ports included — and create_from_memory_snapshot picks up exactly where the episode left off. Useful for long-horizon tasks and branching a live trajectory.
How many rollouts can run at once?
Fleets are created programmatically, so parallelism scales with your training run. Each sandbox is isolated with its own filesystem and processes, and billing is per-second — terminated environments cost nothing.
Can environments have GPUs?
Yes. Pass gpu to the Sandbox constructor for environments that render or run models, and keep CPU-only environments on fractional CPUs so large fleets stay cheap.
Where does the training step run?
On Beam, next to the rollouts. Decorate your update function with @function(gpu=...) and mount the volume your environments write to — rollouts, trajectories, and gradient steps stay on one platform.
Related use cases
Give your AI agents a safe place to run code
Isolated sandboxes for agent-generated code, booting in seconds.
Training & Fine-tuningRun GPU training jobs on-demand
Start training runs on cloud GPUs from Python — pay only while they run.
Training & Fine-tuningFine-tune models on GPUs
LoRA and QLoRA fine-tuning on serverless GPUs.
Start shipping on infra
you won’t outgrow.
Run sandboxes and GPU workloads on your cloud, and scale out to ours when you need to. No infra to manage.