Run your model on RTX 4090 and RTX 5090 GPUs that start when a request arrives and shut down after traffic stops. GPU time starts at $0.69/hr, metered by the millisecond, with nothing to pay once they scale to zero.
from beam import endpoint, Image
@endpoint(
name="my-model",
gpu="RTX4090",
image=Image().add_python_packages(["torch", "transformers"]),
keep_warm_seconds=180,
)
def predict(prompt: str):
return {"output": run_model(prompt)}
# $ beam deploy app.py:predict
# Scales to zero when idle. Metered by the millisecond while it runs.Serverless GPU pricing
A serverless GPU is metered by the millisecond while its container runs, at the per-second rates below. The GPU and the CPU and memory attached to it are metered separately, and billing stops when the endpoint scales to zero.
| GPU | VRAM | Per second | Per hour | Access |
|---|---|---|---|---|
| RTX 4090 | 24 GB | $0.000192 | $0.69 | Every plan |
| RTX 5090 | 32 GB | $0.000303 | $1.09 | Committed-spend agreement |
| CPU and memory | — | $0.000105 per core, $0.0000055 per GiB | $0.379 per core, $0.0198 per GiB | Billed alongside the GPU |
Rates come from the Beam pricing page, which also has a cost calculator.
One RTX 4090 worker
The GPU plus 2 cores and 16 GiB of RAM, charged only while the container runs.
A 5-minute job
The same worker on a task queue, billed for the 5 minutes it runs plus a 10-second keep-warm.
10,000 requests in one daily batch
At 2 seconds of GPU time each, run back to back, plus the 180-second keep-warm before the endpoint scales to zero. Model loading in on_start is billed on top.
When a dedicated machine is cheaper. An on-demand RTX 4090 machine left running all day costs $0.44/hr with 8 vCPU and 64 GB of RAM included, or $10.56 a day. If your GPU would be running more than about 6 hours a day, keep-warm time included, the always-on machine comes out ahead. Below that, serverless costs less because you pay nothing once the endpoint scales to zero.
How serverless GPUs work
A serverless GPU runs your code only when there's work to do. The platform starts a container on demand, scales it out under load, and shuts it down when traffic stops, so you never manage a machine.
- 01
A request arrives
Your app gets an HTTP request, a queued task, or a scheduled run. If no container is warm, Beam starts one on a serverless GPU.
- 02
The container boots from cache
Beam loads your image from its cache, and weights cached in a Volume skip the download, so the container boots in seconds rather than minutes.
- 03
Your code runs on the GPU
Requests go to warm containers, and more containers start when traffic climbs. You pay for the time they run, metered by the millisecond.
- 04
It scales back to zero
Once the keep-warm window passes with no traffic, the container shuts down and billing stops with it.
# 1. Pick a serverless GPU for your function (app.py)
from beam import endpoint, Image
@endpoint(
name="my-model",
gpu="RTX4090",
image=Image().add_python_packages(["torch", "transformers"]),
keep_warm_seconds=180, # stay warm 3 minutes after the last request
)
def predict(prompt: str):
return {"output": run_model(prompt)}
# 2. Deploy it
$ beam deploy app.py:predict
# 3. Call it. The first request starts a container.
$ curl -X POST https://app.beam.cloud/endpoint/my-model \
-H "Authorization: Bearer <TOKEN>" \
-H "Content-Type: application/json" \
-d '{"prompt": "Hello"}'When serverless GPUs make sense
Serverless GPUs pay off when demand is uneven, because billing stops once the endpoint scales to zero. They fit less well when a GPU would stay busy around the clock or a model needs more VRAM than the 32 GB on the largest serverless card, the RTX 5090.
Good fit
- Traffic that spikes, drops to zero overnight, or is hard to predict
- Batch and scheduled jobs that run for minutes, not days
- Prototypes, internal tools, and demos
- Many models that each get light traffic
Poor fit
- Steady load that keeps a GPU busy most of the day
- Long or multi-GPU training runs
- Models that need more VRAM than RTX 4090 and RTX 5090 provide
- Hard latency limits that can't absorb any cold start
Serverless vs on-demand GPUs
Serverless GPUs bill while their containers run, keep-warm time included, and scale to zero. On-demand GPUs are dedicated machines at a flat hourly price that keep running until you stop them. Beam offers both, so you can match each workload to the cheaper model.
| Serverless | On-demand | |
|---|---|---|
| Billing | GPU, CPU, and memory, metered by the millisecond while containers run, keep-warm included | Flat hourly price per machine, CPU, RAM, and NVMe included |
| Idle cost | None once containers scale to zero after the keep-warm window | Billed until you stop the machine |
| Startup | A cold start when no container is warm | None once the machine is up |
| GPUs on Beam | RTX 4090 and RTX 5090 | All 9 GPUs, RTX 4090 to B200, from $0.44/hr |
| Best for | Bursty inference, batch and scheduled jobs | Steady traffic, training, and large models |
GPUs that don't run serverless, such as the H100 and B200, are available as on-demand machines. The pricing page lists each one with its rate. For an always-on machine without a GPU, Beam virtual machines are persistent, CPU-only Linux VMs that can pause and resume.
Serverless GPU cold starts
A cold start is the time it takes to boot a container and load the model when no warm container is available. It's the main latency cost of serverless GPUs. On Beam, containers boot in seconds, and the factors below decide how long the whole cold start takes.
Image size
A large container image takes longer to pull. Beam loads images from its cache, so later boots skip the full download.
Model weights
Downloading weights from a model hub on every boot is often the slowest step. Cache them in a Volume and the container reads them from disk instead.
Loading into GPU memory
Big models take a while to move onto the GPU. Load the model once in on_start and every request after that reuses it.
Keep-warm window
A warm container skips the cold start. Endpoints stay warm for 180 seconds after the last request by default, and that time is billed.
Serverless GPU providers compared
Most serverless GPU platforms bill for the time you use and scale to zero. They differ in which GPUs they offer, what those GPUs cost, and whether you can run the platform yourself.
| Provider | Billing | Scale to zero | Serverless GPUs | Entry GPU rate | Self-host |
|---|---|---|---|---|---|
| Beam | Per millisecond | Yes, after a 180 s keep-warm window by default, billed | RTX 4090 and RTX 5090 | RTX 4090, $0.69/hr | Yes, open source (beta9), without every Beam Cloud feature |
| RunPod | Per second | Yes, 5 s idle timeout | A4000 up to B300 | A4000, $0.58/hr | No |
| Modal | Per second | Yes, 60 s idle window by default | T4 up to B300 | T4, $0.59/hr | No |
| Baseten | Per minute | Yes, 900 s scale-down delay | T4 up to B200 | T4, $0.63/hr | Enterprise option; Baseten runs the control plane |
| Replicate | Per second | Yes, after a few idle minutes | T4, L40S, A100, H100; H200 with committed spend | T4, $0.81/hr | No |
| Cerebrium | Per second | Yes, 10 s cooldown | T4 up to B200 | T4, $0.59/hr | No |
| Fal | Per second | Yes, 60 s keep-alive | A100 up to B200, with account approval | RTX PRO 6000, $4.00/hr; no public A100 or L40 rate | No |
| Google Cloud Run | Per second, 100 ms rounding | Yes, up to 10 min idle, billed | L4, RTX PRO 6000 | L4, $1.05/hr in us-central1, $0.67/hr without zonal redundancy | No |
| Azure Container Apps | Per second | Yes, 300 s cooldown, billed | T4, A100 | T4, $0.26/hr in East US | No; Azure Arc runs Container Apps without serverless GPUs |
Rates are each provider's cheapest serverless GPU with a published list price, so the GPUs differ. CPU, memory, region, and plan tier can add to them, and most platforms, Beam included, bill the idle window before scaling to zero.
What to run on a serverless GPU
Anything that runs in Python and fits on one GPU. Deploy it as an inference endpoint for requests or a task queue for background jobs. These pages cover the most common workloads, with code you can deploy.
Serverless GPU questions
What is serverless GPU hosting?
Serverless GPU hosting runs your code on a GPU only when a request, task, or schedule needs it. The platform starts a container, runs your function, adds containers under load, and shuts them down when traffic stops. You pay for the GPU time you use, metered by the millisecond, instead of an always-on machine.
How does serverless GPU computing work?
Each request goes to a warm container if one is running. If none is, the platform boots one from a cached image and loads the model. That's the cold start. It then runs your code and keeps the container warm for a short window before scaling to zero. Weights baked into the image or cached in a Volume skip the download. Billing covers the time containers are running, warm window included.
Which GPUs can I run serverless on Beam?
RTX 4090 (24 GB, on every plan) and RTX 5090 (32 GB, with a committed-spend agreement). B200, H200, H100, A100 80GB, RTX PRO 6000, L40S and A6000 run as dedicated on-demand machines instead, from $0.54/hr with CPU, RAM, and NVMe included.
Are serverless GPUs cheaper than a GPU VM?
For bursty traffic, usually. You pay only while containers run, so an endpoint that's busy a few hours a day costs a fraction of an always-on machine. Beam's virtual machines are CPU-only, so the always-on GPU option is an on-demand machine: an on-demand RTX 4090 machine left on around the clock costs $0.44/hr, which beats serverless once containers run more than about 6 hours a day, keep-warm time included.
How fast are serverless GPU cold starts?
Beam containers boot in seconds when no warm container is available. Loading a large model adds to that. Caching weights in a Volume and loading them once in on_start keep that part short. A keep-warm window, 180 seconds by default for endpoints, keeps recent containers ready so back-to-back requests skip the cold start. Warm time is billed.
Can I train models on serverless GPUs?
Short fine-tunes and experiments fit well, since billing stops soon after the job ends. Long runs, multi-GPU training, and models that need more than 32 GB of VRAM belong on dedicated on-demand machines such as H100, H200, or B200.
What's the difference between serverless and spot GPUs?
Spot GPUs are discounted spare capacity that the provider can reclaim at any time, and you still manage the machine. Serverless GPUs are managed for you and scale with your traffic. You pay for the time you use rather than for a machine that can be interrupted.
Can I self-host Beam's serverless GPU platform?
Yes. Beam's runtime, beta9, is open source under AGPL-3.0, so you can run it on your own cluster, though it doesn't include every Beam Cloud feature. You can also bring your own cloud account on AWS, GCP, or Azure and let Beam run the control plane.
Give Your Agent Superpowers
Start shipping today and bring your vision to life