PeptideHubBot

The crawler that keeps prices on The Peptide Hub up to date.

  • User agent: PeptideHubBot/1.0 (+https://thepeptidehub.vercel.app/about/bot)
  • Prices: reads public product feeds (Shopify's /products.json, the WooCommerce Store API, or a feed URL a shop gives us). On other platforms it reads product pages listed in the shop's sitemap and takes the price from their schema.org product data.
  • Site checks: once a week it reads the homepage and the pages it links to for delivery, privacy, terms, refunds, contact and lab reports (about ten pages) to work out the Trust Score.
  • Follows robots.txt. Rules for PeptideHubBot take priority over rules for *.
  • One request at a time with a pause between requests. Prices are checked at most every six hours, site checks weekly.
  • Never logs in, adds to basket, or reads customer data.

Stopping or changing it

To stop it, add User-agent: PeptideHubBot and Disallow: / to your robots.txt; your listing stays but prices will show as not collected. To give us a cleaner source, claim your shop and add a product feed.