Request and response format
Information about supported parameters and format of API request.
#
AuthenticationYou can find your API token on the dashboard page in the user's admin panel.
To be able to execute API request add the token as the x-api-key
header to your request.
#
Request formatThe web scraping endpoint is available at the following URL:
It supports both GET
and POST
methods.
#
Passing parameters via GET methodParameters for GET
method should be passed as a query string parameters in urlencoded format.
Example below shows how will look URL with scraping request to example.com
:
#
How to encode query parameters in different programming languages:- Go: url.QueryEscape
- Java: URLEncoder.encode
- NodeJS: encodeURIComponent
- PHP: urlencode
- Python: urllib.parse.quote
- Ruby: URI::escape
#
Passing parameters via POST methodTo pass parameters via POST method you just have to place them into JSON body, like the following:
#
Available parametersBelow you can find the full list of the available parameters:
Parameter | Required | Type | Description |
---|---|---|---|
url | required | String | URL to scrape |
cookies | optional | String | Cookies to pass with a scraping request to the target site. See Custom cookies for more info. |
js_snippet | optional | String | Base64 encoded JS snippet to run once page being loaded in the ScrapingAnt browser. See more at Javascript execution. |
proxy_country | optional | String | Specifies exact country to make request from. See Proxy settings for more info. |
return_text | optional | Boolean | Enables returning text only content from the page instead of full HTML. Default: false |
#
Response structureScrapingAnt web scraping API endpoint returns a JSON object with the content
and cookies
properties:
In case of an error, the response has the HTTP status code in the range of 4xx
or 5xx
and the response properties will be replaced with detail
property.
For example:
See Errors for more details.