Skip to main content
Version: v2

Request and response format

Information about supported parameters and format of API request.

Authentication

You can find your API token on the dashboard page in the user's admin panel.

API parameter-based authentication

The API request can be authenticated using x-api-key query parameter:

curl 'https://api.scrapingant.com/v2/general?url=https%3A%2F%2Fexample.com&x-api-key=<YOUR_SCRAPINGANT_API_KEY>'

Request format

The web scraping endpoint is available at the following URL:

https://api.scrapingant.com/v2/general

Passing parameters via GET method

Parameters should be provided as a query string parameters in urlencoded format.

Example below shows how will look URL with the scraping request to example.com:

https://api.scrapingant.com/v2/general?url=https%3A%2F%2Fexample.com

How to encode query parameters in different programming languages:

Available parameters

Below you can find the full list of the available parameters:

ParameterRequiredTypeDescription
urlrequiredStringURL to scrape
x-api-keyrequiredStringYour ScrapingAnt API key.
browseroptionalBooleanEnables using headless browser for scraping. Default: true. See Headless browser for more info.
return_page_sourceoptionalBooleanEnables returning data returned by the server and unaltered by the browser. Default: false. When true - JS won't be rendered. This feature works only with browser=true. See Scraping without JS rendering for more info.
cookiesoptionalStringCookies to pass with a scraping request to the target site. See Custom cookies for more info.
js_snippetoptionalStringBase64 encoded JS snippet to run once page being loaded in the ScrapingAnt browser. This feature works only with browser=true. See more at Javascript execution.
proxy_typeoptionalStringSpecifies proxy type to make request from. Default: datacenter. See Proxy settings for more info.
proxy_countryoptionalStringSpecifies proxy country to make request from. See Proxy settings for more info.
wait_for_selectoroptionalStringThe CSS selector of the element our service will wait for before returning result. This feature works only with browser=true. See Wait for CSS selector for more info.
block_resourceoptionalString (repeated param)Block resource types. Prevents cloud browser from loading specified resource types. Can be repeated in case several resource types should be blocked. This feature works only with browser=true. Available resource types: document, stylesheet, image, media, font, script, texttrack, xhr, fetch, eventsource, websocket, manifest, other

Response structure

ScrapingAnt web scraping API endpoint returns plain html content of the scraped web page:

<html><body><h1>Hello, World!</h1></body></html>

In case of an error, the response has the HTTP status code in the range of 4xx or 5xx and the response properties will be replaced with detail property.

For example:

{
"detail": "Human readable error description"
}

If you are looking for JSON response format, please check ScrapingAnt's JSON response endpoint.

See Errors for more details.