OpenAI Agents SDK Screenshot Integration

ScreenshotRun integrates with the OpenAI Agents SDK as a @function_tool. With this OpenAI Agents SDK screenshot integration, one decorated async function gives your agent the ability to capture any web page visually, analyze its layout through GPT-4o vision, and return structured observations back to the calling workflow. The entire setup takes about fifteen lines of Python.

The Agents SDK replaced the older Assistants API in August 2026. It runs async by default, supports multi-agent handoffs natively, generates tool schemas from your function signatures, and traces every step in the OpenAI dashboard. That design makes it a natural fit for Python-based screenshot integrations where you need fast, reliable page captures without managing a browser process.

How Tools Work in the Agents SDK

Four types of capabilities exist in the SDK: function tools, hosted tools (like code interpreter), MCP servers, and Computer Use. Function tools are the most common choice for external API calls, and the pattern is simple. You decorate a regular Python function with @function_tool, add type hints to the parameters, and write a docstring. The SDK reads all of that at registration time and builds a JSON schema automatically. When GPT-4o decides it needs your tool, the framework calls your function with the parsed arguments, waits for the return value, and feeds the result back into the conversation.

No manual schema writing — no JSON parsing on your side. The decorator handles serialization in both directions. If your function is async, the SDK awaits it natively since the entire runner loop is built on asyncio.

For the screenshot use case, this means you define one async function that calls the ScreenshotRun API, and the agent gains visual perception of any URL. The docstring you write becomes the tool description that GPT-4o reads when deciding whether to invoke it.

Screenshot Tool with @function_tool

A minimal openai agents sdk screenshot tool needs two things: an async HTTP call to the capture endpoint, and a return value that tells the model what happened. Returning a text description (URL of the saved image, plus metadata) keeps token costs low while still giving the agent enough context to reason about the capture.

import httpx
import os
from agents import function_tool


@function_tool
async def take_screenshot(url: str, full_page: bool = False) -> str:
    """Capture a screenshot of any web page.

    Args:
        url: The web page URL to capture.
        full_page: If True, captures the entire scrollable page.
    """
    async with httpx.AsyncClient(timeout=60) as client:
        params = {
            "url": url,
            "format": "png",
            "width": 1280,
            "height": 720,
            "full_page": str(full_page).lower(),
        }
        resp = await client.get(
            "https://api.screenshotrun.com/v1/screenshots/capture",
            params=params,
            headers={"Authorization": f"Bearer {os.environ['SCREENSHOTRUN_API_KEY']}"},
        )
        resp.raise_for_status()

        # Save locally and return the path
        filename = url.replace("https://", "").replace("/", "_")[:60] + ".png"
        with open(filename, "wb") as f:
            f.write(resp.content)

        return f"Screenshot saved to {filename} ({len(resp.content)} bytes). Page: {url}"

At registration, the SDK reads the type hints (url: str, full_page: bool) and generates the JSON schema that GPT-4o sees. The docstring becomes the tool's description. When your agent runs and encounters a prompt like "show me what google.com looks like," it calls take_screenshot with the right arguments automatically.

Using httpx.AsyncClient keeps everything non-blocking. The Agents SDK runner loop never stalls while waiting for the API response, so other agents in a multi-agent setup can continue processing in parallel.

Returning Screenshots to GPT-4o Vision

Text descriptions work for logging, but sometimes you need the model to actually see the page. GPT-4o supports vision inputs natively, and you can feed it image data by returning a base64 data URI from your tool function. The model receives the screenshot as a visual input and can reason about layout, colors, text content, broken elements, and more.

import httpx
import base64
import os
from agents import function_tool


@function_tool
async def screenshot_for_vision(
    url: str,
    full_page: bool = False,
    block_ads: bool = True,
) -> str:
    """Capture a web page screenshot and return it as an image for visual analysis.

    Args:
        url: The web page URL to screenshot.
        full_page: Capture the full scrollable page instead of just the viewport.
        block_ads: Remove ads before capturing.
    """
    async with httpx.AsyncClient(timeout=60) as client:
        params = {
            "url": url,
            "format": "png",
            "width": 1280,
            "height": 720,
            "full_page": str(full_page).lower(),
            "block_ads": str(block_ads).lower(),
            "response_type": "image",
        }
        resp = await client.get(
            "https://api.screenshotrun.com/v1/screenshots/capture",
            params=params,
            headers={"Authorization": f"Bearer {os.environ['SCREENSHOTRUN_API_KEY']}"},
        )
        resp.raise_for_status()

        b64 = base64.b64encode(resp.content).decode()
        return f"data:image/png;base64,{b64}"

When this tool returns a data URI, GPT-4o processes the embedded image through its vision pipeline. The model can then describe what it sees, flag visual issues, compare layouts across pages, read embedded text, or extract data that appears in the rendered screenshot. This is the same base64 encoding approach used across all vision-capable integrations, but the Agents SDK makes the wiring automatic.

One practical note on token costs: a 1280x720 PNG typically encodes to 200-400 KB of base64, which the model processes as image tokens. For full-page captures of long pages, the image can be several megabytes. If you only need the model to check a specific section, consider using the selector parameter to capture just that element.

Complete Agent Example

Putting the pieces together, a working openai agents screenshot tool needs an agent definition with instructions, the tool binding, a runner call, and a few lines of async boilerplate. Install the SDK first:

pip install openai-agents httpx

Then create the agent:

import asyncio
import os
import httpx
import base64
from agents import Agent, Runner, function_tool


@function_tool
async def take_screenshot(url: str, full_page: bool = False) -> str:
    """Capture a screenshot of a web page and return it for visual analysis.

    Args:
        url: The page URL to capture.
        full_page: Whether to capture the full scrollable page.
    """
    async with httpx.AsyncClient(timeout=60) as client:
        resp = await client.get(
            "https://api.screenshotrun.com/v1/screenshots/capture",
            params={
                "url": url,
                "format": "png",
                "width": 1280,
                "full_page": str(full_page).lower(),
                "block_ads": "true",
                "response_type": "image",
            },
            headers={"Authorization": f"Bearer {os.environ['SCREENSHOTRUN_API_KEY']}"},
        )
        resp.raise_for_status()
        b64 = base64.b64encode(resp.content).decode()
        return f"data:image/png;base64,{b64}"


web_inspector = Agent(
    name="Web Inspector",
    instructions=(
        "You are a visual web analyst. When asked about a website, "
        "capture a screenshot first, then describe what you see. "
        "Focus on layout, branding, calls to action, and any usability issues."
    ),
    tools=[take_screenshot],
)


async def main():
    result = await Runner.run(
        web_inspector,
        "Take a screenshot of https://news.ycombinator.com and tell me what the top 3 stories are about.",
    )
    print(result.final_output)


if __name__ == "__main__":
    asyncio.run(main())

This OpenAI Agents SDK screenshot example produces a conversation where GPT-4o calls take_screenshot, receives the rendered page as an image, and responds with a description of the top stories visible on Hacker News. The entire exchange happens in a single Runner.run call. No manual tool dispatch — no message loop to manage.

Multi-Agent Handoffs with Screenshots

The SDK's most powerful pattern is agent handoffs. Instead of building one monolithic agent with dozens of tools, you split responsibilities across specialized agents that transfer control to each other. A triage agent receives the user's request, decides which specialist should handle it, prepares relevant context, and hands off the conversation.

For screenshot workflows, this means a visual inspector agent handles page captures while other agents focus on text analysis, data extraction, report generation, or notification delivery. Each agent carries only the tools it needs.

from agents import Agent, Runner, function_tool
import asyncio
import httpx
import base64
import os


@function_tool
async def capture_page(url: str, full_page: bool = False) -> str:
    """Capture a visual screenshot of a web page for analysis.

    Args:
        url: Target URL to screenshot.
        full_page: Capture entire scrollable page.
    """
    async with httpx.AsyncClient(timeout=60) as client:
        resp = await client.get(
            "https://api.screenshotrun.com/v1/screenshots/capture",
            params={
                "url": url,
                "format": "png",
                "width": 1280,
                "full_page": str(full_page).lower(),
                "block_ads": "true",
                "response_type": "image",
            },
            headers={"Authorization": f"Bearer {os.environ['SCREENSHOTRUN_API_KEY']}"},
        )
        resp.raise_for_status()
        b64 = base64.b64encode(resp.content).decode()
        return f"data:image/png;base64,{b64}"


visual_agent = Agent(
    name="Visual Inspector",
    instructions=(
        "You inspect web pages visually. Capture screenshots when asked, "
        "then provide detailed analysis of layout, design, content placement, "
        "and any visible errors or accessibility concerns."
    ),
    tools=[capture_page],
)

report_agent = Agent(
    name="Report Writer",
    instructions=(
        "You write concise, structured reports based on observations provided "
        "by other agents. Format findings as bullet points with severity levels."
    ),
)

triage_agent = Agent(
    name="Triage",
    instructions=(
        "You route requests to the right specialist. "
        "For anything involving looking at a website, hand off to Visual Inspector. "
        "For summarizing or formatting findings, hand off to Report Writer."
    ),
    handoffs=[visual_agent, report_agent],
)


async def main():
    result = await Runner.run(
        triage_agent,
        "Check https://example.com for any obvious UX problems and write a short report.",
    )
    print(result.final_output)


asyncio.run(main())

When the triage agent reads the prompt, it recognizes the need for visual inspection and transfers to visual_agent. That agent captures the screenshot, describes what it sees, and the conversation can then flow to report_agent for a formatted summary. Each handoff carries the full message history, so no context is lost between transfers.

This pattern scales cleanly. Add a competitor analysis agent that captures multiple sites for side-by-side comparison. Add a monitoring agent that takes periodic screenshots and flags changes. The triage layer routes traffic, and each specialist stays focused on its own domain.

Computer Use vs Screenshot API

OpenAI's Computer Use tool gives agents access to a full Playwright browser session. The agent can click buttons, fill out forms, navigate between pages, scroll, and interact with dynamic content. It is a powerful capability for tasks that require multi-step browser interaction.

A screenshot API solves a different problem. When you already know the URL and just need a visual capture, launching a full browser session through Computer Use adds latency, cost, resource overhead, and complexity. Computer Use spins up a Playwright instance, loads the page, and then takes a screenshot as one step in a longer interaction chain. ScreenshotRun does the capture in a single API call with sub-second overhead on its end.

In practice, the split looks like this:

Many production systems use both. The Computer Use agent handles interactive browsing when needed, and the screenshot API function tool handles bulk captures of known pages. You can wire both into the same multi-agent setup through separate handoff targets.

Parameters for Agent Screenshots

Agents typically need a focused subset of the full API. These parameters cover the scenarios that come up most often in AI-driven web analysis workflows:

Expose only the parameters your agent actually needs. If it only captures full pages at a fixed viewport, hardcode width and full_page in the function body and keep just url as the tool argument. Fewer parameters mean fewer chances for the model to hallucinate invalid values.

MCP Server Alternative

If you already run a ScreenshotRun MCP server for Claude or other clients, the Agents SDK can connect to it directly. Install the MCP extras and point the agent at your server:

from agents import Agent
from agents.mcp import MCPServerStdio

screenshot_mcp = MCPServerStdio(
    name="ScreenshotRun",
    command="npx",
    args=["-y", "@screenshotrun/mcp-server"],
    env={"SCREENSHOTRUN_API_KEY": "your_key_here"},
)

agent = Agent(
    name="Visual Analyst",
    instructions="Use the screenshot tools to capture and analyze web pages.",
    mcp_servers=[screenshot_mcp],
)

Available tools from the MCP server get discovered at startup and registered automatically. This approach avoids duplicating the HTTP call logic in your function tool code. It also means any updates to the MCP server's capabilities (new parameters, new tools like page analysis) become available to your agent without code changes.

You do add an extra process to manage. For simpler setups where you only need screenshot capture, the @function_tool approach from the earlier sections is lighter weight. For teams that already use MCP across multiple AI clients, the server approach keeps everything consistent.

What Visual Agents Build

Web research agents pull visual context that text-only scraping misses entirely. An agent tasked with competitive analysis can screenshot a competitor's pricing page, read the tier names and prices from the rendered layout, and compare them against your own. The visual capture catches dynamic pricing widgets, JavaScript-rendered tables, and promotional banners that a simple HTTP GET would return as empty div containers.

Pricing monitor agents run on schedules, capturing e-commerce product pages daily and flagging visual changes. Did a competitor drop their price? Add a "Sale" badge? Redesign their product card? A screenshot diff catches these signals faster than parsing unstable HTML selectors that break whenever the site deploys.

Visual regression agents compare staging deployments against production. Before every release, the agent captures both versions of key pages, passes both images to GPT-4o, and asks whether anything looks broken. Missing images, shifted layouts, overflowing text boxes, font rendering issues. The model spots these problems across dozens of pages in the time it takes a human QA tester to open a browser.

Content moderation agents screenshot user-generated pages or embedded content to verify it meets brand guidelines. Social media embeds, forum posts with images, marketplace listings. The agent sees what the end user sees, including dynamically loaded iframes and third-party widgets that server-side rendering ignores.

Portfolio snapshot agents automate the tedious work of maintaining a web agency's case study gallery. Point the agent at a list of client URLs, capture each one at desktop and mobile viewports, and generate a visual report. Clients redesign their sites over time, so periodic recapture keeps your portfolio current without manual effort.

FAQ#