Batch OCR for Large Document Sets on a Serverless GPU

To run batch OCR over a large document set, put an open-source OCR model inside a GPU function, split your files into shards, and call .map() so every shard gets its own container. Each container loads the model once, works through its shard, and writes the extracted text as JSON. You pay for GPU time only while shards are running, and a backlog of 100,000 PDFs needs no cluster.

Run it with python ocr.py. Beam builds the image once, starts a container for each 200-file shard, and streams each shard's summary back as it finishes. The text and layout JSON for every page ends up in the volume next to the source files.

What a batch OCR pipeline has to handle

Batch OCR is mostly a scheduling problem. The model call on one page takes a fraction of a second on a GPU. What decides how long a million pages take, and what the run costs, is how you split the work, how many GPUs you run at once, and what happens when a file fails.

• Ingest. The documents need to sit somewhere every worker can read. A shared volume or a bucket works; copying files into each container one at a time does not.
• Rendering. PDFs become page images before a detection model sees them. Scanned files at high DPI are large, so page rendering is often where memory runs out first.
• The model. A detection model finds text regions and a recognition model reads them. Vision-language models collapse both steps into one and can return Markdown or tables directly.
• Output. Plain text is rarely enough. Keep the word boxes and page structure too, because downstream extraction, search highlighting, and review all need positions.
• Failures. Real document sets contain encrypted PDFs, zero-byte files, and scans with no text. A pipeline that crashes on the first one will never finish a large run.

If your next step is semantic search over the extracted text, the same fan-out pattern runs the embedding stage too. Our guide to batch embeddings at scale picks up where this one ends.

How to run batch OCR on a serverless GPU

Upload the corpus to a volume, wrap the per-document work in a GPU function, and fan out shards with .map(). The steps below cover the details that decide whether a large run finishes cleanly.

1. Copy the documents to a volume. Run beam cp ./pdfs beam://ocr-docs once. Every container in the run mounts the same volume, so no worker has to download its files separately.
2. Size shards so the model load pays off. Loading the model adds startup time to every container. With 100 to 500 documents per shard, that cost is small next to the OCR work. One document per shard spends most of the run starting containers.
3. Wrap each shard in a tuple. .map() unpacks a list item into separate arguments, so passing a list of filenames as the item would call ocr_shard("a.pdf", "b.pdf", ...). Wrapping it as (shard,) passes the whole list as one argument.
4. Catch errors per document, not per shard. The try/except inside the loop records the bad file and moves on. Beam already retries a shard that crashes outright (retries=2 here), but a retry won't fix a corrupt PDF, so catching it inside the loop saves two pointless reruns.
5. Rerun only what failed. The summaries collect every failed path. Fix or skip those files and pass just that list to .map() again, rather than reprocessing the whole set.

For documents that arrive continuously rather than as one backlog, deploy the same function as a task queue. Each upload becomes a task, and a callback_url posts the result to your service when the document is done. The document processing use case shows that setup, and a nightly sweep of new files can run as a scheduled GPU job.

Choosing an OCR model for batch extraction

Pick the smallest model that reads your documents correctly. Classic two-stage OCR models like docTR are fast and cheap on clean printed text, while vision-language models handle messy layouts, tables, and handwriting better at a much higher cost per page.

Run a few hundred representative pages through two candidates before committing to one. Accuracy differences between models are large on some document types and negligible on others, and you can't tell which case you're in from a benchmark. Our roundup of the best OCR models compares them in more depth.

The hero code uses docTR because it installs with pip, runs well on a 24 GB RTX 4090, and returns word-level boxes through result.export(). Swapping in a different model changes only the lines that load and call it.

Batch OCR cost: managed APIs vs a self-hosted GPU

Managed OCR APIs charge per page, and a self-hosted model on a serverless GPU charges per second. Which one is cheaper depends on how many pages a second your model sustains, and the bar is low.

Amazon Textract's text detection, Google Document AI's Enterprise Document OCR, and Azure Document Intelligence's Read model all list $1.50 per 1,000 pages, falling to $0.60 at high monthly volume (above one million pages for Textract and Azure, above five million for Google). That works out to $1,500 for a million pages at the standard rate.

On Beam, a serverless RTX 4090 with 2 physical cores and 16 GiB of RAM costs $1.77 an hour, billed by the millisecond while the function runs. The break-even math:

• To beat $1.50 per 1,000 pages, the container has to process more than 1,180 pages an hour, about one page every three seconds.
• To beat the $0.60 volume rate, it has to process more than 2,950 pages an hour, about one page every 1.2 seconds.
• Cost per 1,000 pages is 1.77 / (pages_per_second × 3.6). A model that holds 2 pages a second comes to about $0.25 per 1,000 pages.

Measure pages per second on your own documents before you budget, since scan resolution and page density move the number a lot. Running 50 containers in parallel doesn't change the cost per page; it makes the run 50 times shorter. The same tradeoff shows up in general batch inference on serverless GPUs.

Cost isn't the only factor. Documents processed on your own model never pass through a third-party OCR service, which matters for contracts, medical records, and anything under a data residency rule.

Batch OCR approaches compared

A managed API is the quickest start, a serverless GPU is usually cheapest once volume grows, and a self-run cluster makes sense mainly for teams that already operate one.

The serverless option scales down to nothing between runs, so a quarterly backfill doesn't leave GPUs idling for the other eleven weeks. For a wider look at how serverless GPU platforms bill and scale, see our serverless GPU guide.

FAQ

How long does it take to OCR a million pages?

It depends on per-page speed and how many containers run in parallel. Divide the total pages by pages per second per GPU, then by the number of shards running at once. A run that would take days on one GPU finishes in hours when .map() spreads it across dozens of containers.

How much does OCR cost per page?

Managed APIs from AWS, Google, and Azure list $1.50 per 1,000 pages at standard volume. Self-hosted OCR on a serverless GPU costs the hourly GPU rate divided by the pages processed in that hour, which falls well below the API price once a model handles more than one page every few seconds.

Is open-source OCR as accurate as Textract or Document AI?

On clean printed documents, open-source models like docTR and PaddleOCR are competitive. On tables, forms, and handwriting, vision-language models such as olmOCR handle layouts that trip up older engines. Test on a sample of your own documents, because results vary a lot by document type.

Can batch OCR extract tables and handwriting?

Yes, but pick the model for it. Two-stage OCR returns words and boxes and leaves table reconstruction to you. Vision-language OCR models can output tables as Markdown or HTML and read handwriting more reliably, at a higher GPU cost per page.

How do I OCR new documents as they arrive instead of in batches?

Deploy the OCR function as a task queue. Each new file becomes a task, containers scale up while tasks are waiting and back to zero when the queue is empty, and a callback URL sends you the result for each document as soon as it's ready.

OCR your whole archive in one run, without a cluster. Get started on Beam.