CAPTCHA and Cloudflare
ScrapingAnt uses anti-scraping technique avoidance mechanisms that drastically increases web scraping success rates.
Browser fingerprinting
In order to define human or robot request websites can use various techniques to test a client's web browser, and store the unique visit attributes to identify repeated visits.
This technology called browser fingerprinting and allows gathering information like downloaded fonts, browser type, operating system, extensions, timezone, etc. to create a unique fingerprint of the browser to restrict further access.
Our service always uses only the latest versions of real browsers which are running in the real-world execution environment.
Regional IP blocking
Websites can also use regional IP blocking to restrict data access for some countries and regions.
ScrapingAnt's proxy country setting allows API consumer to choose the right country for the region-restricted web scraping.
Session data check
A popular option some websites use is blocking access based on existence of session data, as particular web pages can't be accessed without passion of defined user-flow.
ScrapingAnt's JSON response endpoint always contains cookies data, and we're recommending to use this data while scraping to chain requests and prevent data restrictions.
Check out our custom cookies documentation to know more about available API request options.