#robots.txt for bookmarkit.org #Wildcard robots.txt agent (ask all bots to crawl slowly but still index everything). User-agent: * Crawl-delay: 1 Disallow: