In this article
  • How Pinterest accesses your site
  • Restrict or limit Pinterest from accessing your site
  • To help people discover and do what they love, we're creating a database of billions of Pins on Pinterest. In order to protect our users and provide the highest quality content, we use web crawlers to help us identify the data on the pages behind the Pins.

    These pages contain rich signals that enable us to infer better recommendations, fight spam and display useful information. To take full advantage of these signals, we regularly fetch, store and process page content associated with Pins.​

    How Pinterest accesses your site

    When a genuine Pinterest crawler visits your website, it will send a valid Pinterest user agent and connect from a network operated by Pinterest.

    The Pinterest crawler respects the Robots Exclusion Standard (robots.txt) and is configured to rate limit concurrent requests made to your site. Do not hardcode these IP addresses in your site configuration, because the addresses that the crawler uses can change without notice.

    Pinterest's user agent is:
    Pinterest/0.2 (+https://www.pinterest.com/bot.html)
    Mozilla/5.0 (compatible; Pinterestbot/1.0; +https://www.pinterest.com/bot.html)
    Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Pinterestbot/1.0; +https://www.pinterest.com/bot.html)

    Pinterest's IP is dynamic and and changes frequently, but will always be in the range of: 54.236.1.XXX.

    Geo-distributed crawling

    Pinterest crawler primarily crawls with US-based IP addresses. We may also crawl with IP addresses based outside the US to support localised data.

    For US-based IP addresses, they’re always in the range of: 54.236.1.XXX.

    For non US-based IP addresses, there’s no fixed range. To verify the authenticity of the Pinterest crawler, we recommend that you follow the steps in the next section as the preferred way, rather than relying on IP range check. This is useful if you're concerned that spammers are accessing your site while faking to be Pinterestbot.

    Verify the Pinterest crawler
  • Use the host command to run a reverse DNS lookup on the IP address from your logs
  • Verify that the domain name in the response ends with pinterest.com or pinterestcrawler.com
  • Use the host command to run a forward DNS lookup on the response retrieved from step 1
  • Verify that it's the same as the IP address from step 1
  • If you receive a consistent volume of traffic from a client sending a valid Pinterest user agent but it does not pass the above DNS test, please contact us.

    Restrict or limit Pinterest's access to your site

    To modify the behaviour of the Pinterest crawler, you'll need to update your site's robots.txt file. Make sure you place the robots.txt file on your main domain, because we do not support robots.txt files on subdomains.

    Large crawl delays affect the distribution and recommendation of your content on Pinterest. Therefore, we honour delays of up to 1 and will treat any larger values as 1.  If you acknowledge this and still need a larger crawl delay, contact us.

    user agent: Pinterestbot
    Crawl delay: 0.2

    Large crawl delays may negatively affect the distribution or recommendation of your site content on Pinterest. We honour delays of up to 1. We'll treat any larger values as 1.

    user agent: Pinterestbot
    disallow: /

     

    user agent: Pinterestbot
    disallow: /directory1/
    user agent: Pinterestbot
    disallow: /directory1/
    allow: /directory1/subdirectory1/
    user agent: Pinterestbot
    disallow: /directory1/
    allow: /directory1/subdirectory1/
    crawl delay: 0.2
    End of Other articles Links
    Still need help? Contact us
    User feedback
    Was this article helpful?

    collection_fields

    How can we make this article better?