Pinterest crawler

To help people discover and do what they love, we're creating a database of billions of Pins on Pinterest. In order to protect our users and provide the highest quality content, we use web crawlers to help us identify the data on the pages behind the Pins.

These pages contain rich signals that enable us to infer better recommendations, fight spam, and display useful information. To take full advantage of these signals, we regularly fetch, store, and process page content associated with Pins.

How Pinterest accesses your site

When a genuine Pinterest crawler visits your website, it will send a valid Pinterest User-Agent and connect from a network operated by Pinterest. In addition to respecting the Robots Exclusion Standard, the Pinterest crawler is configured to automatically rate limit concurrent requests made to your website.

We recommend that webmasters avoid hard-coding these network's IP addresses in their site configuration, as the addresses that the crawler uses may change without notice.

Pinterest's user agent is:

Pinterest/0.2 (+https://www.pinterest.com/bot.html)
Mozilla/5.0 (compatible; Pinterestbot/1.0; +https://www.pinterest.com/bot.html)
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Pinterestbot/1.0; +https://www.pinterest.com/bot.html)

Pinterest's IP is dynamic and thus constantly changing, but will always be in the range of: 54.236.1.XXX.

Verify the Pinterest crawler

  1. Use the host command to run a reverse DNS lookup on the IP address from your logs
  2. Verify that the domain name in the response ends with pinterest.com
  3. Use the host command to run a forward DNS lookup on the response retrieved from step 1
  4. Verify that it is the same as the IP address from step 1

If you receive a consistent volume of traffic from a client sending a valid Pinterest User-Agent but it does not pass the above DNS test, please open a support ticket.

Restrict Pinterest from accessing your site

To modify the behavior of the Pinterest crawler, you'll need to update your site's robots.txt file. The Pinterest crawler obeys the following directives:

  • Disallow
  • Allow
  • Crawl-delay
     
Still need help?
Contact us