MCP Server for AI Assistants

Enable your AI assistant to scrape web pages using ScrapingAnt's MCP server.

What is MCP?

Model Context Protocol (MCP) is an open standard that enables AI assistants to connect with external data sources and tools. ScrapingAnt's MCP server allows AI assistants like Claude, Cursor, and VS Code Copilot to scrape web pages directly during conversations.

Authentication

The MCP server accepts two kinds of credentials. Both use the same URL: https://api.scrapingant.com/mcp.

• Sign in with your ScrapingAnt account (OAuth). Clients that support OAuth for remote MCP servers, such as Claude Desktop, claude.ai, Claude Code and VS Code, open the ScrapingAnt dashboard in your browser. You click Allow and the client receives its own token. Your API key is never copied into a config file.
• API key. Clients that only support fixed headers send your key in the x-api-key header. Find it on the dashboard.

note

A sign-in is bound to the API key that was active when you clicked Allow. If you regenerate the key in the dashboard, re-authenticate the MCP server in your client.

Getting Started

Claude Desktop and claude.ai

Claude Desktop and claude.ai add remote MCP servers as connectors. claude_desktop_config.json only works for local servers.

1. Open Settings → Connectors and click Add custom connector.
2. Enter a name (for example ScrapingAnt) and the URL https://api.scrapingant.com/mcp. Leave the client ID and secret fields empty.
3. Click Add, then Connect. A browser tab opens the ScrapingAnt dashboard: log in if needed and click Allow.
4. In a chat, open the tools menu and enable the ScrapingAnt connector.

On Team and Enterprise plans an organization owner adds the connector under Organization settings → Connectors first; members then connect individually.

Claude Code (CLI)

claude mcp add --transport http scrapingant https://api.scrapingant.com/mcp

Then run /mcp inside a Claude Code session and choose Authenticate to sign in with your ScrapingAnt account.

To use the API key instead:

claude mcp add --transport http scrapingant https://api.scrapingant.com/mcp -H "x-api-key: <YOUR-API-KEY>"

VS Code / GitHub Copilot

Create .vscode/mcp.json in your workspace, or add the server to your user settings:

{
  "servers": {
    "scrapingant": {
      "type": "http",
      "url": "https://api.scrapingant.com/mcp",
      "headers": {
        "x-api-key": "<YOUR-API-KEY>"
      }
    }
  }
}

Omit headers to sign in with your ScrapingAnt account instead; VS Code prompts you to complete the authorization in the browser.

Cursor

1. Open Settings → MCP → Add new MCP Server
2. Enter the following configuration:
   • Name: scrapingant
   • URL: https://api.scrapingant.com/mcp
   • Transport: streamableHttp
   • Headers: x-api-key: <YOUR-API-KEY>

Claude Code (CLI)

Run the following command in your terminal:

claude mcp add scrapingant --transport http https://api.scrapingant.com/mcp -H "x-api-key: <YOUR-API-KEY>"

Cline

Add the following to your cline_mcp_settings.json:

{
  "mcpServers": {
    "scrapingant": {
      "url": "https://api.scrapingant.com/mcp",
      "transport": "streamableHttp",
      "headers": {
        "x-api-key": "<YOUR-API-KEY>"
      }
    }
  }
}

Windsurf

Follow the Windsurf MCP documentation using the standard configuration:

{
  "mcpServers": {
    "scrapingant": {
      "url": "https://api.scrapingant.com/mcp",
      "transport": "streamableHttp",
      "headers": {
        "x-api-key": "<YOUR-API-KEY>"
      }
    }
  }
}

Available Tools

The ScrapingAnt MCP server provides three tools for web scraping:

Tool	Description
get_web_page_html	Fetch a URL and return raw HTML content
get_web_page_markdown	Fetch a URL and return content as Markdown
get_web_page_text	Fetch a URL and return plain text content

Parameters

All three tools accept the same parameters:

Parameter	Type	Required	Default	Description
url	string	✅	-	The URL of the page to scrape
browser	boolean		true	Whether to use browser rendering
proxy_type	string		datacenter	Proxy type: datacenter or residential
proxy_country	string		Random	ISO-3166 country code. See available proxy countries

Parameter Details

browser

By default, ScrapingAnt uses a headless browser to render pages. This is essential for JavaScript-heavy websites and Single-Page Applications (SPAs). Set to false for lightweight HTML parsing of static pages.

proxy_type

• datacenter (default): High-speed datacenter proxies. Best for most use cases.
• residential: Premium residential proxies. Use when encountering anti-bot detection for improved success rates.

proxy_country

Specify a country code to route requests through proxies in that region. Useful for accessing geo-restricted content or testing location-specific behavior.

Usage Examples

Once configured, you can ask your AI assistant to scrape web pages naturally:

Basic Scraping

"Fetch the content from https://example.com and summarize it"

"Get the HTML from https://news.ycombinator.com"

Using Markdown Output

"Scrape https://docs.python.org/3/tutorial/index.html as markdown and explain the main topics"

With Residential Proxies

"Fetch https://example.com using residential proxies"

Geo-targeted Requests

"Get the content from https://example.com using a proxy from Germany"

Credits Usage

MCP server requests consume API credits the same way as regular API calls. Learn more about credits and pricing.