Technical guide · reviewed September 2026

Headless browser vs HTTP requests: when to use each

A practical decision guide for extracting public web data

The response you fetch is not always the page a visitor sees. A server-rendered document can contain the data in its initial HTML. A JavaScript application may return an app shell first and fill it after scripts run. Choosing the smallest tool that contains the data keeps your integration easier to operate.

The trade-off at a glance

Inspect the raw response first

Use an HTTP client when the required data is already in the response, such as an RSS feed, JSON endpoint, server-rendered article, or static product markup.

const response = await fetch('https://demo.example/products/widget');
const html = await response.text();
// Parse with an HTML parser in your application; this fixture is synthetic.
console.log(html.includes('Widget Pro'));

A response that contains only <div id="root"></div> and script references is evidence that you may need a browser for the content you want. It is not proof that every page in the application requires one.

Use browser rendering for client-rendered content

Choose a browser when a normal public browser must execute JavaScript before the required text, metadata, or JSON-LD appears. ExtractAPI's authenticated endpoint accepts javascript, an optional waitFor CSS selector, and an optional selector whose text is returned in data.custom.

curl -X POST https://extractapi.app/v1/extract \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://demo.example/products/widget",
    "javascript": true,
    "waitFor": "[data-ready=\"true\"]",
    "selector": "[data-price]"
  }'

The service returns bounded fields such as title, metaDescription, h1, headings, mainContent, structuredData, optional custom, and page statistics under data. A rendered response is still an observation of a public page at a point in time.

A safe hybrid decision

1. Identify the exact field and target pages you are allowed to read.
2. Try the simplest permitted representation first: an upstream API, feed, or raw HTML.
3. When the field is client-rendered, use browser mode for that request and wait for a stable selector if one is documented.
4. Validate missing fields, status, and normalized output; do not silently treat an empty result as success.
5. Bound retries and preserve a sanitized error category so selector drift and target outages can be diagnosed.

// Conceptual routing; keep parsing and retries in your application.
async function extractTarget(target) {
  const raw = await fetch(target.url);
  const html = await raw.text();
  if (target.fieldIsInInitialHtml && html.includes(target.marker)) {
    return parseHtml(html);
  }
  return callExtractApi({
    url: target.url,
    javascript: true,
    waitFor: target.waitFor
  });
}

What this does not promise

• JavaScript rendering does not guarantee success for every SPA, third-party widget, login flow, challenge, or protected domain.
• Browser mode does not make a target's terms, robots rules, or privacy obligations disappear. Obtain permission and use reasonable rates.
• ExtractAPI is not a screenshot service and does not promise pixel-identical sessions; non-essential resources may be bounded or blocked.
• Compare your own success rate and latency by target and request shape. The page contains no universal benchmark or SLA claim.

FAQ

Get a free API key