Skip to main content

Request and response format

Information about supported parameters and format of API request.

Authentication#

You can find your API token on the dashboard page in the user's admin panel.

To be able to execute API request add the token as the x-api-key header to your request.

Request format#

The web scraping endpoint is available at the following URL:

https://api.scrapingant.com/v1/general

It supports both GET and POST methods.

Passing parameters via GET method#

Parameters for GET method should be passed as a query string parameters in urlencoded format.

Example below shows how will look URL with scraping request to example.com:

https://api.scrapingant.com/v1/general?url=https%3A%2F%2Fexample.com

How to encode query parameters in different programming languages:#

Passing parameters via POST method#

To pass parameters via POST method you just have to place them into JSON body, like the following:

{
"url": "https://example.com"
}

Available parameters#

Below you can find the full list of the available parameters:

ParameterRequiredTypeDescription
urlrequiredStringURL to scrape
browseroptionalBooleanEnables using headless browser for scraping. Default: true. See Headless browser for more info.
cookiesoptionalStringCookies to pass with a scraping request to the target site. See Custom cookies for more info.
js_snippetoptionalStringBase64 encoded JS snippet to run once page being loaded in the ScrapingAnt browser. This feature works only with browser=true. See more at Javascript execution.
proxy_typeoptionalStringSpecifies proxy type to make request from. Default: datacenter. See Proxy settings for more info.
proxy_countryoptionalStringSpecifies proxy country to make request from. See Proxy settings for more info.
wait_for_selectoroptionalStringThe CSS selector of the element our service will wait for before returning result. This feature works only with browser=true. See Wait for CSS selector for more info.
return_textoptionalBooleanEnables returning text only content from the page instead of full HTML. Default: false

Response structure#

ScrapingAnt web scraping API endpoint returns a JSON object with the content and cookies properties:

{
"content": "<html><body><h1>Hello, World!</h1></body></html>",
"cookies": "cookie_name_1=cookie_value_1;cookie_name_2=cookie_value_2"
}

In case of an error, the response has the HTTP status code in the range of 4xx or 5xx and the response properties will be replaced with detail property.

For example:

{
"detail": "Human readable error description"
}

See Errors for more details.