Skip to main content
Version: v2

Request and response format

Information about supported parameters and format of API request.

Authentication

You can find your API token on the dashboard page in the user's admin panel.

API parameter-based authentication

The API request can be authenticated using x-api-key query parameter:

curl 'https://api.scrapingant.com/v2/general?url=https%3A%2F%2Fexample.com&x-api-key=<YOUR_SCRAPINGANT_API_KEY>'

Request format

The web scraping endpoint is available at the following URL:

https://api.scrapingant.com/v2/general

Passing parameters via GET method

Parameters should be provided as a query string parameters in urlencoded format.

Example below shows how will look URL with the scraping request to example.com:

https://api.scrapingant.com/v2/general?url=https%3A%2F%2Fexample.com

How to encode query parameters in different programming languages:

Available parameters

Below you can find the full list of the available parameters:

ParameterRequiredTypeDescription
urlrequiredStringURL to scrape
x-api-keyrequiredStringYour ScrapingAnt API key.
browseroptionalBooleanEnables using headless browser for scraping. Default: true. See Headless browser for more info.
cookiesoptionalStringCookies to pass with a scraping request to the target site. See Custom cookies for more info.
js_snippetoptionalStringBase64 encoded JS snippet to run once page being loaded in the ScrapingAnt browser. This feature works only with browser=true. See more at Javascript execution.
proxy_typeoptionalStringSpecifies proxy type to make request from. Default: datacenter. See Proxy settings for more info.
proxy_countryoptionalStringSpecifies proxy country to make request from. See Proxy settings for more info.
wait_for_selectoroptionalStringThe CSS selector of the element our service will wait for before returning result. This feature works only with browser=true. See Wait for CSS selector for more info.
return_page_sourceoptionalBooleanEnables returning HTML returned by the server and unaltered by the browser. Default: false. This feature works only with browser=true.

Response structure

ScrapingAnt web scraping API endpoint returns a JSON object with the content, cookies and status_code properties:

{
"content": "<html><body><h1>Hello, World!</h1></body></html>",
"cookies": "cookie_name_1=cookie_value_1;cookie_name_2=cookie_value_2",
"status_code": 200
}
ParameterTypeDescription
contentStringContent from the scraped web page
cookiesStringResponse cookies from the scraped web page
status_codeNumberStatus code received from the scraped page

In case of an error, the response has the HTTP status code in the range of 4xx or 5xx and the response properties will be replaced with detail property.

For example:

{
"detail": "Human readable error description"
}

See Errors for more details.