Identity
User agent PickentraBot/1.0. When configured, it appends a link to our /about page, for example PickentraBot/1.0 (+https://<our-site>/about).
Crawler
PickentraBot/1.0
The crawler that reads a business website when that business asks Pickentra to verify it.
Facts
Identity, purpose and scope
Purpose
Initial customer-authorized review of a business website to extract a draft profile for customer confirmation. That confirmation is customer attestation, not independent proof of website ownership. It is not a general web crawler.
Scope
At most 20 page-attempt slots per verification, including retries, within a 60 second limit and 512 KB per response.
Boundaries
HTTPS only, same site as the submitted website, public network addresses only. Private, loopback, link-local and metadata addresses are rejected. Redirects are rechecked, up to 5.
robots.txt
Rules for PickentraBot are honored. Crawl-delay is respected within the crawl deadline; non-finite delays or delays over five seconds stop the check as unavailable. A missing robots.txt (404/410) allows crawling; 401/403 is treated as excluded; other errors stop the crawl as unavailable.
No forms or logins
It makes GET requests only and does not submit forms or sign in.
Exclusion
How to exclude PickentraBot
Add a rule to your robots.txt. Disallowed pages are not fetched.
User-agent: PickentraBot Disallow: /
You can also email info@pickentra.com. There is no automated opt-out system, and we cannot promise a specific response time.