Use case · AI agents and integrations

MCP web extraction for AI agents

Agents often need the current text and metadata from a public page, but an MCP client should not have to know how to parse every site. ExtractAPI's local MCP server exposes one tool, extract, over the standard stdio transport.

Who this is for

This workflow fits developers building research, support, browser-assistance, or data-enrichment agents that read public web pages. It is a connector for public-page extraction, not a way to access private accounts or defeat a site's access controls.

Two modes, one tool

1. Lightweight mode: run the server without EXTRACTAPI_KEY. It fetches the public URL, removes script and style blocks, bounds the response, and returns a title, text, HTTP status, and content type. It does not execute JavaScript.
2. Browser mode: set EXTRACTAPI_KEY. The same extract tool calls POST /v1/extract and can request browser rendering with javascript, wait for a CSS selector with waitFor, or return a selected element with selector.

Install and configure

Use the repository checkout; the package name in mcp/package.json is a local binary entry point, not a claim that a public npm package has been published.

cd /path/to/extractapi/mcp
npm install
# Lightweight mode
node server.js

# Browser mode (POSIX)
EXTRACTAPI_KEY=YOUR_API_KEY node server.js

# Browser mode (PowerShell)
$env:EXTRACTAPI_KEY = "YOUR_API_KEY"
node .\server.js

For local development, set EXTRACTAPI_BASE_URL=http://localhost:3000 beside the key. The default base URL is https://extractapi.app.

Claude Desktop or another stdio client

{
  "mcpServers": {
    "extractapi": {
      "command": "node",
      "args": ["/path/to/extractapi/mcp/server.js"],
      "env": {
        "EXTRACTAPI_KEY": "YOUR_API_KEY"
      }
    }
  }
}

The path is local to your machine. Store the real key in the client’s environment configuration and keep that file out of source control.

Input and output

This synthetic tool call asks for a browser-rendered product page. The URL and response are examples only; no live extraction is performed by this page.

// MCP tools/call arguments
{
  "name": "extract",
  "arguments": {
    "url": "https://demo.example/products/widget",
    "javascript": true,
    "waitFor": "[data-ready='true']"
  }
}

// Browser-mode text content returned by the MCP server
{
  "url": "https://demo.example/products/widget",
  "title": "Widget Pro | Demo Store",
  "content": "Widget Pro\nIn stock\n$49.00",
  "status": "ok",
  "contentType": null,
  "responseTimeMs": 842
}

In browser mode, the REST response also contains structured fields under data, including metaDescription, headings, mainContent, structuredData, and optional links or images. The MCP adapter intentionally returns a smaller, flat text shape.

Supported implementation steps

1. Start in lightweight mode and confirm that the target is a public page your use case is allowed to read.
2. Issue a key through the normal signup flow when an agent needs JavaScript rendering or CSS selectors.
3. Set the key only in the MCP process environment and use javascript:true for pages whose useful content appears after client-side rendering.
4. Pass a narrow waitFor selector when the page documents a stable ready element; treat a missing selector as a signal to inspect the result, not as proof that a page is fully loaded.
5. Keep the agent’s downstream schema separate from the raw extraction response so missing fields and blocked targets remain observable.

Limitations and safe boundaries

• Lightweight mode cannot see content that exists only after JavaScript runs. Browser mode can still fail when a page is unavailable, too large, slow, protected, or dependent on interactions outside the supported request.
• ExtractAPI does not promise access to every site, CAPTCHA solving, login automation, or bypassing robots rules, consent gates, or other controls. Check the target’s terms and applicable law.
• Use a public, sanitized target in development. Never send API keys, cookies, authorization headers, extracted private data, or verification tokens as target content.
• The local MCP server uses a bounded fetch path without a credential and a backend path with the API key. A successful connection to the MCP process is not proof that a target extraction will succeed.

FAQ

Get a free API key