Migrate ScrapingBee HTML requests to ScrapingAnt

Move one synchronous HTML workflow first: keep your parser, select explicit request modes and validate the output before changing your production queue. For plan limits and mode economics, see the ScrapingBee alternative comparison.

Eligibility checklist

Good candidates are independent read-only pages where your application already schedules requests and parses HTML. Paid ScrapingAnt plans have no plan concurrency cap; keep an application worker limit and raise it only after output and budget checks. Free allows one concurrent request and has domain restrictions.

Inventory these dependencies before changing providers:

Current ScrapingBee dependency	Migration decision
Plain or rendered HTML	Use /v2/general; set browser explicitly. Keep and test your HTML parser.
json_response or response metadata	Use /v2/extended only after rewriting your response adapter; schemas differ.
session_id for a stable IP for five minutes	No documented equivalent in this API. Passing cookies does not preserve an IP or browser session. Retain the old integration if this is required.
Screenshots, including full-page or selector capture	No documented screenshot parameter in this API. Keep that workflow separate.
own_proxy	No documented bring-your-own-proxy parameter here. Datacenter and residential select ScrapingAnt-managed pools.
mode=auto and max_cost	No direct mapping. Choose a fixed mode and enforce an application budget. Do not silently escalate modes on retries.
extract_rules, ai_query, evaluate_results, or dedicated search/product APIs	Design and validate a new output schema. /v2/extended provides a page envelope, not a replacement extraction schema. AI extraction is a separate integration with additional credit costs.
Stealth routing or a multi-step browser scenario	Treat this as a separate evaluation. Neither a proxy label nor a renamed JSON scenario establishes equivalence.

This guide covers the HTML API, not all ScrapingBee products. Required features may favor keeping your existing integration. ScrapingAnt's standalone proxy products are separate from this API; their sessions and GB pricing do not establish API parity.

Endpoint and authentication

ScrapingBee authenticates with an Authorization: Bearer header (its api_key query parameter is deprecated). ScrapingAnt uses the x-api-key parameter with your ScrapingAnt key from the dashboard, so drop the Bee header.

Before, with explicit HTTP mode:

curl --get 'https://app.scrapingbee.com/api/v1/' --max-time 75 \
  --header "Authorization: Bearer ${SCRAPINGBEE_API_KEY}" \
  --data-urlencode 'url=https://example.com/?q=tea&sort=asc' \
  --data-urlencode 'render_js=false' \
  --data-urlencode 'premium_proxy=false' \
  --data-urlencode 'timeout=30000'

After, with the same target URL:

curl --get 'https://api.scrapingant.com/v2/general' --max-time 45 \
  --data-urlencode "x-api-key=${SCRAPINGANT_API_KEY}" \
  --data-urlencode 'url=https://example.com/?q=tea&sort=asc' \
  --data-urlencode 'browser=false' \
  --data-urlencode 'proxy_type=datacenter' \
  --data-urlencode 'timeout=30'

Each command makes one authenticated request if run and can consume credits. Both APIs enable rendering by default, so retain explicit booleans. Bee's timeout=30000 is in milliseconds; Ant's timeout=30 is in seconds. Keep keys, cookies and private target URLs out of logs and shared exception objects.

Map parameters by behavior

ScrapingBee parameter	ScrapingAnt alternative	Required change
render_js	browser	Set false for direct HTTP, true for rendering.
country_code	proxy_country	Validate supported country codes; use e.g. US. Bee keeps separate country lists for classic and premium proxies; every country in Ant's list works with both datacenter and residential proxies, at the normal credit cost for that proxy type, with no geotargeting surcharge.
premium_proxy	proxy_type=residential where needed	A similar functional proxy class, not an identical pool or access guarantee. Default Ant routing is datacenter.
stealth_proxy	No direct equivalent	Bee's stealth tier only works with JavaScript rendering and does not support custom headers or cookies. Do not translate it to residential as a parity claim; test an Ant browser/proxy configuration deliberately.
CSS wait_for	wait_for_selector	Set browser=true. Rewrite XPath selectors as CSS, for example //div[@id='results'] becomes #results. Check the new selector on your page.
JSON js_scenario	Rewritten Base64 js_snippet	Rewrite actions, waits and result checks as JavaScript. Never Base64-encode the JSON scenario and expect it to execute.
forward_headers / raw API Spb-* headers	Raw API ant-* headers	Remove the flag. Prefix only intended target headers, e.g. ant-Accept-Language. Do not forward provider credentials. SDK conventions differ.
cookies	cookies	Supply name=value;name2=value2 through a query builder. Only names and values carry over; inspect domain/path/expiry needs separately.
return_page_source	return_page_source	In Ant this requires browser=true and returns original server HTML. Its credit rate differs from direct HTTP and rendered DOM.
timeout	timeout plus client deadline	Bee uses milliseconds; Ant uses seconds in the range 5–60. Convert rather than copying the number: 30000 ms becomes 30 seconds. Bee timeouts above 60000 ms have no Ant equivalent. Keep the client limit longer than the API timeout.
Resource, device and window settings	Review individually	Do not copy flags blindly; check documented request options.

Rewrite a scenario deliberately

Bee's js_scenario={"instructions":[{"scroll_y":500},{"wait":1000}]} can become the following scroll-and-wait script, passed through the get_response helper from bounded retries. It only illustrates those two actions; it does not prove a page has finished loading.

import base64
import os

script = "window.scrollBy(0, 500); await new Promise(r => setTimeout(r, 1000));"
response = get_response(
    "https://example.com/?q=tea&sort=asc",
    os.environ["SCRAPINGANT_API_KEY"],
    browser=True,
    params={"js_snippet": base64.b64encode(script.encode("utf-8")).decode("ascii")},
)
print("Credits charged:", response.headers.get("Ant-credits-cost"))

Use selector waiting and explicit DOM completion checks for the actual page. Snippets that submit forms or change state must not use automatic GET retries. Read the JavaScript execution reference.

Headers, cookies and structured query parameters

For a browser request requiring .product, these builders select the extended endpoint so your adapter can inspect target status, headers and cookies. They construct options without making a request. Install Python requests; use Node.js 22+ with built-in fetch.

# Build options; this function does not send a request.
def request_options(url, api_key):
    return {
        "params": {
            "url": url, "x-api-key": api_key,
            "browser": "true", "proxy_type": "datacenter",
            "timeout": 30, "wait_for_selector": ".product",
            "cookies": "locale=en;currency=USD",
        },
        "headers": {"ant-Accept-Language": "en-US"},
        "timeout": (5, 45),
        "allow_redirects": False,
    }

// Build options; this function does not send a request.
function requestOptions(url, apiKey) {
  const endpoint = new URL('https://api.scrapingant.com/v2/extended');
  endpoint.search = new URLSearchParams({
    url, 'x-api-key': apiKey, browser: 'true', proxy_type: 'datacenter',
    timeout: '30', wait_for_selector: '.product',
    cookies: 'locale=en;currency=USD',
  }).toString();
  return {endpoint, options: {
    headers: {'ant-Accept-Language': 'en-US'},
    signal: AbortSignal.timeout(45000), redirect: 'manual',
  }};
}

Pass a full unencoded target URL to these builders; they encode query values, including cookies, once. Do not pass the Bee Authorization header through as ant-Authorization. See custom headers and cookies.

Adapt and validate responses

For /v2/general, use response.text and the target Ant-page-status-code header. For /v2/extended, parse JSON html and integer status_code; cookies is a string and headers is a list of name/value objects. There is no Bee extraction-result schema in that envelope. General responses expose original target headers under ant-original-header-*; use extended JSON when you need explicit target headers and cookies.

These adapters accept only target 2xx HTML containing a required marker. Choose checks meaningful to your own parser, such as product IDs, field types and row counts; the class="product" marker below is an example. Handle intentional target 404s separately if your application needs them.

def accept_html(api_status, target_status, html, required_text):
    if api_status != 200:
        raise ValueError(f"API HTTP {api_status}; diagnose before parsing")
    if type(target_status) is not int or not 200 <= target_status < 300:
        raise ValueError("Unexpected target status")
    if not isinstance(html, str) or not required_text or required_text not in html:
        raise ValueError("Expected content missing; keep this result out of the parser")
    return html

Use with the request builder above (a single attempt, without automatic retries):

import os
import requests

try:
    response = requests.get(
        "https://api.scrapingant.com/v2/extended",
        **request_options("https://example.com/products", os.environ["SCRAPINGANT_API_KEY"]),
    )
except requests.RequestException:
    raise RuntimeError("Transport failure; check the outcome before retrying") from None
if response.status_code != 200:
    raise ValueError(f"API HTTP {response.status_code}; inspect detail privately")
page = response.json()  # Stop on malformed JSON; never resubmit just because parsing failed.
if not isinstance(page, dict):
    raise ValueError("Unexpected JSON envelope; expected an object")
html = accept_html(response.status_code, page.get("status_code"), page.get("html"), 'class="product"')
# Validate page['cookies'] and page['headers'] before adapting your session storage.
# Record response.headers.get('Ant-credits-cost') without logging credentials.

function acceptHtml(apiStatus, targetStatus, html, requiredText) {
  if (apiStatus !== 200) throw new Error(`API HTTP ${apiStatus}; diagnose before parsing`);
  if (!Number.isInteger(targetStatus) || targetStatus < 200 || targetStatus >= 300) {
    throw new Error('Unexpected target status');
  }
  if (typeof html !== 'string' || !requiredText || !html.includes(requiredText)) {
    throw new Error('Expected content missing; keep this result out of the parser');
  }
  return html;
}

In the same .mjs file as requestOptions and acceptHtml:

const {endpoint, options} = requestOptions(
  'https://example.com/products', process.env.SCRAPINGANT_API_KEY,
);
let response, body;
try {
  response = await fetch(endpoint, options);
  body = await response.text();
} catch {
  throw new Error('Transport failure; check the outcome before retrying');
}
if (response.status !== 200) throw new Error(`API HTTP ${response.status}; inspect detail privately`);
const page = JSON.parse(body); // Stop on malformed JSON.
if (page === null || typeof page !== 'object' || Array.isArray(page)) {
  throw new Error('Unexpected JSON envelope; expected an object');
}
const html = acceptHtml(response.status, page.status_code, page.html, 'class="product"');
// Adapt page.cookies / page.headers separately; record the Ant-credits-cost header.

To use the adapters with /v2/general, convert its numeric target-status header to an integer, rejecting missing or malformed values first. Feed that status and HTML to the same acceptance check.

Bounded retries and failure diagnosis

Check a representative target's content, cookies and status before switching production traffic. For each chosen configuration, record useful outputs, total charged credits including retries, and latency. Compare credits per useful result rather than treating every outer 200 as a useful page.

Keep the previous provider configuration available behind a provider-selection setting. Move a small batch of safe reads first; if expected fields, output quality or the credit budget fail your checks, stop the new queue and route subsequent work back to the previous integration. Avoid sending duplicate state-changing operations during a switch.

A successful /v2/general request returns outer HTTP 200 with the target status in Ant-page-status-code. On /v2/extended, check the outer HTTP status first, then read the target's JSON status_code and html. A target 403 or 404 in those fields is a separate result, not an API authentication error. See response formats and errors.

Use this retry policy for reads that are safe to repeat:

Outer API result	Action
403	Stop. Inspect JSON detail locally for an invalid key, exhausted quota or IP restriction; fix that condition first.
404	ScrapingAnt could not reach the site or the worker timed out (unreachable site). This is not a target 404 in Ant-page-status-code. The helpers stop here: check the URL's reachability, optional snippets/selectors and the API timeout before sending the request again.
409	Queue free-plan work sequentially. A small delayed retry budget can handle an overlap.
423	Detection: retry within a small budget, then review settings deliberately.
500	A small delayed retry budget may handle a temporary server failure.
Redirects, validation errors, other statuses, network failure or client timeout	Stop and diagnose. A timeout does not prove the server finished.

The examples below make at most three GET attempts, with increasing delays and jitter for outer 409, 423 and 500 only. Every attempt sends the same browser and proxy mode, extra API parameters and headers. Set browser=True / browser: true explicitly when rendering is needed. Review credit costs before choosing residential or rendering; there is no automatic mode upgrade.

ScrapingAnt charges no credits for a request when the ScrapingAnt API itself returns a non-200 status (failed, detected or any other error), so a retry that ends in an API error costs nothing. An outer 200 is charged even if the target page inside it is a 403, 404 or otherwise unusable. Each response's Ant-credits-cost header reports the credits spent; add it up across attempts to track the real cost of a job.

Python: install requests with python -m pip install requests, then save this helper in your script:

import random
import time
import requests

API_TIMEOUT = 30  # ScrapingAnt execution limit in seconds (documented range 5-60).
CLIENT_TIMEOUT = (5, API_TIMEOUT + 15)  # Connect and read timeouts; read must exceed API_TIMEOUT.


class ScrapingAntError(RuntimeError):
    def __init__(self, response, attempt):
        self.status_code = response.status_code
        try:
            self.detail = response.json().get("detail")
        except (ValueError, AttributeError):
            self.detail = None
        super().__init__(f"ScrapingAnt HTTP {self.status_code} after {attempt} attempt(s)")


def get_response(url, api_key, *, browser=False, proxy_type="datacenter", params=None, headers=None):
    query = {
        **(params or {}),
        "url": url,
        "x-api-key": api_key,
        "browser": str(browser).lower(),
        "proxy_type": proxy_type,
        "timeout": API_TIMEOUT,
    }
    request_headers = dict(headers or {})
    for attempt in range(1, 4):
        try:
            response = requests.get(
                "https://api.scrapingant.com/v2/general",
                params=query,
                headers=request_headers,
                timeout=CLIENT_TIMEOUT,
                allow_redirects=False,
            )
        except requests.RequestException:
            raise RuntimeError("Transport failure or timeout; check the outcome before retrying") from None
        if response.status_code == 200:
            return response
        error = ScrapingAntError(response, attempt)
        response.close()
        if error.status_code not in (409, 423, 500) or attempt == 3:
            raise error
        time.sleep(2 ** (attempt - 1) + random.uniform(0, 0.25))


def page_html(response):
    # Requests assumes ISO-8859-1 for text/* without a charset; default to UTF-8 instead.
    has_charset = "charset=" in response.headers.get("Content-Type", "").lower()
    encoding = response.encoding if has_charset and response.encoding else "utf-8"
    return response.content.decode(encoding, errors="replace")

Set SCRAPINGANT_API_KEY in your environment, then call the helper. Pass other API parameters, such as proxy_country, wait_for_selector or js_snippet, in params (use a list, such as {"block_resource": ["image", "font"]}, to repeat a parameter), and ant- prefixed target headers in headers:

import os

response = get_response(
    "https://example.com/?q=tea&sort=asc",
    os.environ["SCRAPINGANT_API_KEY"],
    browser=False,
    params={"proxy_country": "US"},
    headers={"ant-Accept-Language": "en-US"},
)
print("Target status:", response.headers.get("Ant-page-status-code"))
print("Credits charged:", response.headers.get("Ant-credits-cost"))
html = page_html(response)
# Check the target status and expected content before passing html to your parser.

JavaScript (Node.js 22+): built-in fetch needs no package installation. Save this helper in a .mjs file:

const API_TIMEOUT = 30; // ScrapingAnt execution limit in seconds (documented range 5-60).
const CLIENT_TIMEOUT_MS = (API_TIMEOUT + 15) * 1000; // Must exceed API_TIMEOUT.

async function getResponse(url, apiKey, {browser = false, proxyType = 'datacenter', params = {}, headers = {}} = {}) {
  const endpoint = new URL('https://api.scrapingant.com/v2/general');
  const fixed = {url, 'x-api-key': apiKey, browser: String(browser), proxy_type: proxyType, timeout: String(API_TIMEOUT)};
  const query = new URLSearchParams();
  for (const [name, value] of Object.entries(params)) {
    if (name in fixed) continue;
    for (const item of [].concat(value)) query.append(name, String(item)); // Arrays repeat the parameter.
  }
  for (const [name, value] of Object.entries(fixed)) query.set(name, value);
  endpoint.search = query.toString();
  const requestHeaders = {...headers};

  for (let attempt = 1; attempt <= 3; attempt++) {
    let response, body;
    try {
      response = await fetch(endpoint, {
        method: 'GET',
        headers: requestHeaders,
        signal: AbortSignal.timeout(CLIENT_TIMEOUT_MS),
        redirect: 'manual',
      });
      body = await response.text();
    } catch {
      throw new Error('Transport failure or timeout; check the outcome before retrying');
    }
    if (response.status === 200) {
      return {
        html: body,
        targetStatus: response.headers.get('Ant-page-status-code'),
        creditsCost: response.headers.get('Ant-credits-cost'),
      };
    }
    const error = new Error(`ScrapingAnt HTTP ${response.status} after ${attempt} attempt(s)`);
    error.status = response.status;
    try { error.detail = JSON.parse(body).detail; } catch { /* Non-JSON gateway or redirect response. */ }
    if (![409, 423, 500].includes(response.status) || attempt === 3) throw error;
    await new Promise(resolve => setTimeout(resolve, (2 ** (attempt - 1) + Math.random() / 4) * 1000));
  }
}

With SCRAPINGANT_API_KEY set, add this call and run the file with node your-file.mjs:

const result = await getResponse(
  'https://example.com/?q=tea&sort=asc',
  process.env.SCRAPINGANT_API_KEY,
  {browser: false, params: {proxy_country: 'US'}, headers: {'ant-Accept-Language': 'en-US'}},
);
console.log('Target status:', result.targetStatus);
console.log('Credits charged:', result.creditsCost);
// Check targetStatus and expected content before passing result.html to your parser.

The query builder encodes the full target URL once, preserving its own &, +, % and fragment characters. API_TIMEOUT is sent as the timeout query parameter and limits API execution in seconds; the documented range is 5–60 seconds. The client limit is derived from it (API_TIMEOUT + 15) and must always be longer, otherwise your client can give up on a request that ScrapingAnt still completes and charges. Change API_TIMEOUT rather than passing timeout in params. Python's read timeout measures inactivity rather than a total wall-clock deadline; the JavaScript signal covers each attempt including body consumption.

A 3xx response from the API is treated as a non-retryable status error in both languages: Python disables redirect following and JavaScript uses redirect: 'manual'.

The error objects retain detail for diagnosis, while their messages omit the key and request URL. Redact detail before logging or sharing it. Avoid logging full requests or exception objects containing credentials or private target data.

Do not reuse this GET retry loop for state-changing POST, PUT, PATCH or DELETE requests, or for browser snippets that submit forms or otherwise change state. A timeout or server error does not establish whether the target operation already happened. Confirm its outcome or use a target-supported idempotency mechanism before repeating it. See request methods and safe retries.

Validate success, latency and billing on your own workload.

Budget, canary and rollback

Compare charged credits per accepted result, including retries and rejected content. Nominal credits are not interchangeable. Record the Ant-credits-cost header from every response, including retries. Ant's ordinary datacenter rates are 1/2/10 for HTTP/browser page-source/rendered HTML; residential rates are 25/50/125. Target exceptions apply, including Google datacenter at 10. Bee premium HTTP/rendered costs 10/25 and its separate stealth tier costs 75; these modes do not imply equivalent access. See the plan comparison and Ant credit table.

1. Keep your existing Bee credentials, request settings and response adapter behind a provider-selection setting. Define acceptance fields, target-status policy, maximum retry count, worker limit and credit budget before testing.
2. Start with a small eligible read-only canary, with explicit browser/proxy modes. Free-plan tests must be sequential; they cannot validate paid concurrency. Evaluate cookies, language/country, parser output and total credits for representative targets.
3. Increase traffic only when the agreed output and budget checks pass. Inspect 403 details privately; do not retry a key, quota or IP-access failure. A 423 indicates detection and may use the shared bounded retry policy without a mode upgrade.
4. If content, target status, errors or credit use violate your acceptance budget, pause the Ant queue and route subsequent work back through the previous adapter. Investigate in a separate sample. Do not double-submit state-changing work or blindly resubmit timed-out requests.

FAQ

Does ScrapingAnt bill for unsuccessful requests?

ScrapingAnt charges no credits for a request when the ScrapingAnt API itself returns a non-200 status (failed, detected or any other error). An outer HTTP 200 is charged even when the target page inside it is a 403, 404 or otherwise unwanted, so inspect its status and content. See credits.

References and scope

Official references checked October 18, 2026: ScrapingBee HTML API, ScrapingBee plans, ScrapingAnt parameters, response envelope, errors and credit costs. Examples use raw HTTP conventions; SDK interfaces may differ.

For a workflow-specific question, contact support@scrapingant.com with redacted settings and the required output. Keep provider keys and private page content out of shared reports.