PeptideHubBot
The crawler that keeps prices on The Peptide Hub up to date.
- User agent:
PeptideHubBot/1.0 (+https://thepeptidehub.vercel.app/about/bot) - Prices: reads public product feeds (Shopify's
/products.json, the WooCommerce Store API, or a feed URL a shop gives us). On other platforms it reads product pages listed in the shop's sitemap and takes the price from their schema.org product data. - Site checks: once a week it reads the homepage and the pages it links to for delivery, privacy, terms, refunds, contact and lab reports (about ten pages) to work out the Trust Score.
- Follows
robots.txt. Rules forPeptideHubBottake priority over rules for*. - One request at a time with a pause between requests. Prices are checked at most every six hours, site checks weekly.
- Never logs in, adds to basket, or reads customer data.
Stopping or changing it
To stop it, add User-agent: PeptideHubBot and Disallow: / to your robots.txt; your listing stays but prices will show as not collected. To give us a cleaner source, claim your shop and add a product feed.