Bulk Screenshot API

At some point, every developer building a screenshot feature hits the same wall: the code works fine for ten URLs, maybe fifty, but somewhere around a few hundred it falls apart. Chromium starts leaking memory, tabs pile up faster than they close, and the server grinds to a halt. The jump from "I can screenshot a page" to "I can screenshot ten thousand pages" is where most self-hosted solutions quietly break. A bulk screenshot API exists specifically for that jump. You bring the URL list, the API handles the browsers.

Why bulk screenshot workflows break at a few hundred URLs

The problem is almost always Chromium. Each headless browser tab allocates somewhere between 150 and 500 MB of RAM depending on the page. A simple marketing site might sit at the low end. A JavaScript-heavy dashboard with third-party analytics, embedded maps, and a chat widget can push past 400 MB easily. Run twenty tabs in parallel on a 4 GB server and you're one heavy page away from the kernel's OOM killer shutting down your process.

Memory isn't even the hardest part. Chromium leaks memory over time. Open and close a thousand tabs through Puppeteer and you'll notice the process footprint creeping upward even after all tabs are closed. By the time you've processed 500 URLs, the browser instance is sluggish. By 1,000 it's unreliable. The standard workaround is restarting Chromium every few hundred pages, but that adds startup overhead and forces you to build state management around a process that wasn't designed to be restarted mid-job.

Then there's the failure cascade. One URL that hangs (maybe a server that accepts the connection but never sends a response) can block the entire pipeline if your concurrency pool fills up. Without proper timeout handling, retry logic, and dead-letter queuing, a single bad URL at position 347 in your list can stall everything behind it. I've debugged pipelines where a team's nightly screenshot job ran fine for weeks and then failed silently because one site started returning 30-second SSL handshakes. Nobody noticed until the monitoring dashboard was showing screenshots from three weeks ago.

And scaling beyond a single server means either running Selenium Grid, managing a cluster of Chromium containers in Kubernetes, or building your own queue system with workers. That's real infrastructure engineering for what should be a straightforward task: give me an image of this URL.

How to capture thousands of screenshots with an API

Instead of running Chromium locally, you send HTTP requests to a screenshot API and get images back. Each request is independent. No shared browser state, no memory accumulation, no zombie processes. The API manages its own Chromium pool, handles retries internally, and scales concurrency without you thinking about it.

For bulk capture, the key is parallelizing those requests. A Node.js example that processes a list of URLs with controlled concurrency:

const fs = require('fs');

const API_KEY = process.env.SCREENSHOTRUN_API_KEY;
const CONCURRENCY = 10;

const urls = [
  'https://example.com',
  'https://github.com',
  'https://stripe.com',
  // ... hundreds more
];

async function capture(url) {
  const params = new URLSearchParams({
    url,
    format: 'webp',
    width: '1280',
    height: '800',
    block_cookies: 'true',
    block_ads: 'true'
  });

  const res = await fetch(
    `https://api.screenshotrun.com/v1/screenshots/capture?${params}`,
    { headers: { 'Authorization': `Bearer ${API_KEY}` } }
  );

  if (!res.ok) throw new Error(`${url}: ${res.status}`);

  const filename = url.replace(/https?:\/\//, '')
    .replace(/[^a-z0-9]/gi, '-') + '.webp';
  fs.writeFileSync(`./screenshots/${filename}`, Buffer.from(await res.arrayBuffer()));
  return filename;
}

async function processAll(urls, concurrency) {
  const results = [];
  for (let i = 0; i < urls.length; i += concurrency) {
    const batch = urls.slice(i, i + concurrency);
    const settled = await Promise.allSettled(batch.map(capture));
    results.push(...settled);
    console.log(`Processed ${Math.min(i + concurrency, urls.length)}/${urls.length}`);
  }
  return results;
}

processAll(urls, CONCURRENCY);

Ten concurrent requests, each independent. If one URL fails, the others still complete. No shared browser, no process cleanup. The Node.js integration page covers the full parameter set if you need to customize viewport, device type, or output format per URL.

Same idea in Python with asyncio:

import asyncio
import aiohttp
import os

API_KEY = os.environ['SCREENSHOTRUN_API_KEY']
CONCURRENCY = 10

urls = [
    'https://example.com',
    'https://github.com',
    'https://stripe.com',
    # ... hundreds more
]

async def capture(session, semaphore, url):
    async with semaphore:
        params = {
            'url': url,
            'format': 'webp',
            'width': '1280',
            'height': '800',
            'block_cookies': 'true',
            'block_ads': 'true',
        }
        headers = {'Authorization': f'Bearer {API_KEY}'}
        async with session.get(
            'https://api.screenshotrun.com/v1/screenshots/capture',
            params=params, headers=headers
        ) as resp:
            if resp.status == 200:
                filename = url.replace('https://', '').replace('/', '-') + '.webp'
                with open(f'./screenshots/{filename}', 'wb') as f:
                    f.write(await resp.read())
                return filename
            raise Exception(f'{url}: {resp.status}')

async def main():
    semaphore = asyncio.Semaphore(CONCURRENCY)
    async with aiohttp.ClientSession() as session:
        tasks = [capture(session, semaphore, url) for url in urls]
        results = await asyncio.gather(*tasks, return_exceptions=True)
        ok = sum(1 for r in results if not isinstance(r, Exception))
        print(f'Done: {ok}/{len(urls)} succeeded')

asyncio.run(main())

Full parameter reference is on the Python integration page. For quick one-off bulk jobs, a bash loop with cURL works too:

#!/bin/bash
while IFS= read -r url; do
  filename=$(echo "$url" | sed 's|https\?://||;s|[^a-zA-Z0-9]|-|g').webp
  curl -s "https://api.screenshotrun.com/v1/screenshots/capture?url=${url}&format=webp&width=1280&block_cookies=true" \
    -H "Authorization: Bearer $SCREENSHOTRUN_API_KEY" \
    -o "./screenshots/${filename}" &

  # Limit concurrency to 5 parallel requests
  [ $(jobs -r | wc -l) -ge 5 ] && wait -n
done < urls.txt
wait
echo "Done"

Feed it a text file with one URL per line. Five parallel captures, no dependencies beyond cURL. I used a version of this script to capture 800 competitor pages for a client's market analysis last quarter. Not elegant, but it finished in under twenty minutes and the output was good enough for the deck.

Async delivery with webhooks

Every example above uses synchronous requests — you wait for each screenshot to come back before processing it. That works well for up to a few thousand URLs. Beyond that, or when individual pages take ten-plus seconds to render, holding HTTP connections open starts getting wasteful.

The webhook parameter changes the pattern. Instead of waiting for the image, the API accepts the request, queues it, and POSTs the result to your endpoint when it's ready. Your code fires off all requests immediately and processes results as they arrive:

// Fire-and-forget pattern with webhooks
async function captureAsync(url) {
  const params = new URLSearchParams({
    url,
    format: 'png',
    full_page: 'true',
    block_cookies: 'true',
    webhook_url: 'https://yoursite.com/api/screenshot-callback'
  });

  await fetch(
    `https://api.screenshotrun.com/v1/screenshots/capture?${params}`,
    { headers: { 'Authorization': `Bearer ${API_KEY}` } }
  );
}

// Fire all at once — no waiting
urls.forEach(captureAsync);

Your callback endpoint receives each screenshot as it finishes. No open connections, no timeout management, no concurrency tracking on your side. This is the pattern that scales to tens of thousands of URLs without your server caring about how long each page takes to render.

Who uses bulk screenshots (and what they capture)

SEO agencies audit client sites by capturing every indexed page at mobile and desktop viewports. A 500-page site audit that took days of manual screenshotting becomes a ten-minute script. The SEO audit page covers this workflow in detail, including metadata extraction alongside captures.

Marketing teams use bulk capture for competitor monitoring — screenshotting competitor landing pages weekly to track design changes, messaging shifts, and pricing updates. Ten competitors with twenty pages each is 200 screenshots a week. Not enough to justify infrastructure, too many to do by hand.

Archiving services build timestamped evidence libraries. Legal teams, compliance departments, and journalists capture web pages as visual records. The website archiving use case goes deeper on retention and full-page capture for complete page evidence.

E-commerce teams screenshot product catalogs after every deployment to catch visual regressions — broken images, misaligned prices, or layout shifts that only show up on specific product pages. Running visual regression test across 2,000 product pages catches bugs that no amount of unit testing would find.

Directory and marketplace platforms generate website thumbnails for every listing. A SaaS directory with 1,000 tools needs 1,000 thumbnail previews, and those previews need refreshing when the listed sites redesign. Monthly bulk recapture keeps the directory looking current.

What to look for in a bulk screenshot API

Not every screenshot API handles volume gracefully. Some are built for single captures and just happen to allow you to call them in a loop. The things that actually matter when you're processing thousands of URLs at a time might surprise you.

Webhook delivery matters most. If the API only supports synchronous responses, your client has to manage timeouts and retries for every single request. With webhooks, you fire off requests and process results as they land. That's the difference between a pipeline that scales and one that chokes on slow pages.

Caching saves both money and time. If you capture the same URL twice within a time window (maybe the same site appears in multiple client reports, or you're re-running a failed batch), a cache_ttl parameter returns the cached result instantly without burning a new capture. Over thousands of URLs with some overlap, the savings add up.

Content blocking matters more at scale than for single captures. One cookie banner on one screenshot is annoying. Cookie banners on 500 screenshots make the entire batch unusable for client-facing reports. Blocking consent overlays, ad scripts, and live chat pop-ups at the API level means you don't write per-site dismissal logic. And format flexibility lets you optimize per use case: WebP for web-ready thumbnails, PNG for pixel-perfect archiving, PDF for legal evidence. Choosing the right format at capture time avoids a post-processing conversion step.

ScreenshotRun's free tier handles 200 screenshots per month — enough to test a bulk workflow against your actual URL list before committing. If you're coming from a self-hosted Puppeteer setup, the Puppeteer comparison covers what changes in the migration and what stays the same.