Skip to main content
Version: v2

Javascript execution

How to execute your Javascript code in the scraped page context.

Executing custom Javascript​

You can pass your custom Javascript code to run in the browser context using js_snippet param, and it will be executed after the page load will finish.

Custom javascript can be used for interaction with a page, like scrolling, pressing a button, etc.

Your Javascript snippet must be Base64 encoded.

To encode your JS snippet into the Base64 encoded string, you can use the following tools:

import base64
base64_snippet = base64.b64encode("JS SNIPPET".encode()).decode()
const buffer = new Buffer('JS SNIPPET');
let base64_snippet = buffer.toString('base64');
$base64_snippet = base64_encode('JS SNIPPET');

Those examples are only the most commonly used ones. To find out how to do it for your programming language, just Google base64 encoding <your_programming_language>

JS execution time​

Every URL fetching time is limited to 60 seconds, so be aware of it while writing your Javascript snippet.

Examples​

Scroll to the bottom of the page and wait 2 seconds.​

window.scrollTo(0,document.body.scrollHeight);
await new Promise(r => setTimeout(r, 2000));

To execute this Javascript after the page load, you have to encode it using Base64 encoding and add it to the API request.

The API request that scrapes example.com and executes scrolling to the bottom of the page after the page load looks like the following:

https://api.scrapingant.com/v2/general?url=https%3A%2F%2Fexample.com&js_snippet=d2luZG93LnNjcm9sbFRvKDAsZG9jdW1lbnQuYm9keS5zY3JvbGxIZWlnaHQpOwphd2FpdCBuZXcgUHJvbWlzZShyID0+IHNldFRpbWVvdXQociwgMjAwMCkpOw==

Click a button by ID​

document.getElementById('button-id').click();

Below you can find a helpful tool that makes Base64 encoding for you.

Base64 data encoding​

Please, enter your JS code you'd like to run after the page load:

The following result can be inserted in the API request as Base64 encoded JS snippet: