Extract Clickable Elements from Any Web Page — Buttons, Links, Inputs & ARIA Controls

Every web page has dozens of things you can click, tap, or type into — buttons, links, form fields, dropdown menus, tabs, checkboxes. Finding all of them programmatically usually means launching a headless browser, writing CSS selectors, and parsing the DOM yourself. Or you send one POST request. Pass include: ["interactive"] to the analyze endpoint and you get back every interactive element on the page: tag name, visible text, type, ARIA attributes, and a pixel-precise bounding rectangle. No browser to manage, no selectors to maintain.

curl -X POST https://api.screenshotrun.com/v1/analyze \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "include": ["interactive"]
  }'

What comes back is a flat array of every interactive element the browser found. Links carry their href. Inputs carry their type and name. ARIA-labeled elements carry their label. Custom components with role="button" or role="tab" show up alongside native HTML elements. And every entry includes a rect (four integers that tell you exactly where that element sits on the page).

What counts as a clickable element

Our renderer scans for twelve selectors that cover the full spectrum of interactive HTML:

• a[href] — links with a destination
• button — native buttons
• input, select, textarea — form controls
• [role="button"], [role="link"] — ARIA buttons and links (common in React, Vue, Angular apps)
• [role="tab"], [role="checkbox"], [role="radio"], [role="switch"] — custom controls
• [role="menuitem"] — dropdown and context menu entries

This catches elements that simpler approaches miss. A <div role="button" tabindex="0"> looks like a div in the DOM but behaves like a button for users and screen readers. A <span role="link"> inside a JavaScript framework routes pages without a real anchor tag. Static HTML parsing skips both. Our renderer catches them because it queries the same attributes the browser uses for focus and interaction.

Data returned for each element

{
  "interactive": [
    {
      "tag": "a",
      "text": "Pricing",
      "href": "/pricing",
      "rect": [680, 24, 64, 40]
    },
    {
      "tag": "button",
      "text": "Get Started",
      "role": "button",
      "rect": [920, 24, 140, 44]
    },
    {
      "tag": "input",
      "type": "email",
      "name": "email",
      "label": "Email address",
      "rect": [400, 320, 480, 44]
    },
    {
      "tag": "select",
      "name": "country",
      "label": "Select your country",
      "rect": [400, 380, 480, 44]
    },
    {
      "tag": "div",
      "role": "tab",
      "text": "Monthly",
      "expanded": false,
      "rect": [200, 500, 120, 40]
    }
  ]
}

Fields appear conditionally. A link gets href but no type. An input gets type and name but probably no href. The label field pulls from aria-label — useful for inputs where the visible placeholder text isn't the same as the accessible name. And expanded tracks aria-expanded, so you know whether a dropdown or accordion is currently open.

Text is capped at 100 characters. Long button labels or link text gets truncated, but 100 characters is enough to identify any element. If you need the full text, the element's position in rect lets you target it with a follow-up scrape.

How AI agents use clickable element maps

A browser agent needs three things before it can act on a web page: what's clickable, what each clickable thing does, and where it sits on screen. Raw HTML answers the first two (poorly, and at 50k–200k tokens of context), but says nothing about pixel position. Accessibility trees are smaller but still miss spatial data. Screenshots give position but lose semantics.

The interactive category delivers all three in one compact payload. A typical page with 30 interactive elements produces about 1,500 tokens of JSON. Compare that to the same page's raw HTML at 80,000 tokens (a 98% reduction that leaves room for the agent's actual reasoning).

import requests

response = requests.post(
    "https://api.screenshotrun.com/v1/analyze",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "url": target_url,
        "include": ["interactive"],
        "stealth": True,
    },
    timeout=60,
)

elements = response.json()["result"]["interactive"]

# Build a concise action menu for the agent
action_menu = []
for el in elements:
    label = el.get("text") or el.get("label") or el.get("name") or el["tag"]
    x, y, w, h = el["rect"]
    action_menu.append(f"{label} [{el['tag']}] at ({x + w//2}, {y + h//2})")

# Feed to LLM: "Here are the clickable elements on the page: ..."
prompt = "Available actions:\n" + "\n".join(action_menu)

Using the ScreenshotRun MCP Server? Claude gets interactive element data via analyze_page out of the box. No wrapper code needed on your side.

Finding all outbound links on a page

Filter the interactive array to links and you get every outbound URL on the page. No regex against raw HTML, no Beautiful Soup parser, no missed JavaScript-rendered links.

links = [
    el for el in data["result"]["interactive"]
    if el["tag"] == "a" and el.get("href")
]

# Separate internal vs external
from urllib.parse import urlparse
page_domain = urlparse(target_url).netloc

internal = [l for l in links if urlparse(l["href"]).netloc in ("", page_domain)]
external = [l for l in links if l not in internal]

print(f"{len(internal)} internal links, {len(external)} external links")
for link in external:
    print(f"  {link['text'][:50]} → {link['href']}")

This handles JavaScript-rendered navigation that static scrapers can't see. A React app might generate its entire menu at runtime. No <a> tags appear in the server's HTML output. Since our renderer executes JavaScript before scanning, every dynamically created link shows up in the result.

Mapping button inventory across a site

Run the analysis across multiple pages to build an inventory of every button and CTA on your site. Spot inconsistencies: is "Sign Up" sometimes "Register" and sometimes "Create Account"? Do some pages have no primary CTA at all?

const urls = ["/", "/pricing", "/features", "/docs"];
const buttonMap = {};

for (const path of urls) {
  const res = await fetch("https://api.screenshotrun.com/v1/analyze", {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.SCREENSHOTRUN_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      url: `https://yoursite.com${path}`,
      include: ["interactive"],
    }),
  }).then(r => r.json());

  const buttons = res.result.interactive.filter(
    el => el.tag === "button" || el.role === "button"
  );
  buttonMap[path] = buttons.map(b => ({
    text: b.text,
    position: b.rect,
    aboveFold: b.rect[1] + b.rect[3] <= 800,
  }));
}

// Report: which pages lack an above-fold CTA?
for (const [page, buttons] of Object.entries(buttonMap)) {
  const aboveFold = buttons.filter(b => b.aboveFold);
  if (aboveFold.length === 0) {
    console.warn(`${page}: no buttons above the fold`);
  }
}

Marketing teams use this to audit CTA placement across landing pages. Product teams use it to verify that key actions are accessible on every page. And it runs in minutes, not the hours a manual audit would take.

Detecting JavaScript-rendered interactive elements

Static HTML scraping (curl, wget, requests without a browser) misses interactive elements created by JavaScript frameworks. A Vue component that renders a <button> after mounting, a React portal that creates a modal with form controls, an Angular directive that adds role="button" at runtime — none of these show up in the raw HTML you'd get from a simple GET request.

Our analyze endpoint loads pages in Chromium and waits for JavaScript to finish executing. Components mount, API calls resolve, dynamic menus render. The interactive elements you get back are the ones a real user would see and click. For sites with aggressive bot protection, enable stealth mode to bypass headless browser checks.

Filtering by element type

The API returns all interactive elements in one array. Client-side filtering narrows it down. Need only form inputs? Filter by tag === "input". Only links? Filter by tag === "a". Only custom ARIA components? Filter by the role field.

# Group elements by type
from collections import defaultdict

by_type = defaultdict(list)
for el in data["result"]["interactive"]:
    key = el.get("role") or el["tag"]
    by_type[key].append(el)

for element_type, elements in sorted(by_type.items()):
    print(f"{element_type}: {len(elements)} elements")
    for el in elements[:3]:
        print(f"  '{el.get('text', el.get('name', ''))[:40]}' at {el['rect']}")

Server-side, you can narrow the scope with the scope parameter (a CSS selector that limits analysis to one container) and visible_only (default true, set to false for hidden elements). Between these two parameters and client-side filtering, you can slice the interactive inventory any way you need.

Security: finding suspicious clickables

Phishing pages and compromised sites often add invisible interactive elements — clickjacking overlays, transparent buttons that sit on top of legitimate content, hidden links that redirect to malicious domains. Combine visible_only: false with outbound link analysis to surface these:

# Scan for suspicious interactive elements
hidden_clickables = [
    el for el in data["result"]["interactive"]
    if el["rect"] == [0, 0, 0, 0] or (el["rect"][2] < 2 and el["rect"][3] < 2)
]

for el in hidden_clickables:
    href = el.get("href", "")
    if href and not href.startswith("/"):
        print(f"SUSPICIOUS: hidden {el['tag']} pointing to {href}")
        print(f"  Text: '{el.get('text', '')[:60]}', rect: {el['rect']}")

A legitimate site rarely has invisible links pointing to external domains. When defacement monitoring or phishing detection flags a page, this analysis gives you the structural evidence: which elements are hidden, where they point, and what they pretend to be.

Try it yourself

Grab a free API key and send a POST to /v1/analyze with any URL. The Playground lets you test interactively — paste a URL, select the interactive category, and see every clickable element mapped out with its coordinates. No code required to start exploring. When you're ready to integrate, the API docs cover every parameter, response field, and code example in Python, Node.js, Go, and PHP.

FAQ#