GET
/
api
/
v2
/
website
/
rendering

If you need to scrape a website, Piloterr’s Website Crawler API is the perfect solution. It’s a high-performance API that can scrape any website and return the HTML content. You can use it to scrape any website, including those with JavaScript rendering, and bypass Cloudflare, Akamai, PerimeterX, and DataDome. The API is easy to use and requires only a website URL and an API key. You can also set parameters to control the API’s behavior, such as following redirects, waiting for JavaScript to render, and waiting for specific elements to appear in the DOM. The API is perfect for web scraping, data extraction, and content analysis.

This endpoint costs 2 credits per request. Contact us if you need help or have any questions.

For security and ethical reasons, Piloterr implements a comprehensive filtering system for websites accessible through this endpoint. This measure helps prevent misuse and ensures compliance with legal and ethical standards. If you wish to use this endpoint, please contact our support team to request the addition of your domain to our allowlist. We’ll review each request carefully, considering factors such as content appropriateness, legal compliance, and potential impact on our infrastructure. This process helps maintain the quality and reliability of our service for all users.

x-api-key
string
required

This parameter specifies the private key you’ll need for Piloterr access.

​
Parameters

query
string

A website URL with either the http or https protocol.

wait_in_seconds
integer

Some code-heavy websites need time to fully “render”. To direct Piloterr to wait before it returns the fully rendered HTML, use the wait parameter with a value in seconds between 0 and 20. The Piloterr headless browsers will then wait the duration of the time set in seconds before returning the page’s HTML.

wait_for
string

It’s sometimes necessary to wait for a particular element to appear in the DOM before Piloterr returns the HTML content. Our headless browsers will wait for the CSS / Xpath selector passed in the parameter before returning the HTML. For example:

  • To wait for an element with ID: wait_for=#loading-container
  • To wait for an element with class: wait_for=.content-loaded
  • To wait for a specific element: wait_for=div.main-content#user-profile
block_ads
boolean

By default, Piloterr does not block ads. Set this parameter to true to block ads.

​
Response

Was this page helpful?

Website CrawlerWebsite Technology