# www.robotstxt.org/ # Allow crawling of all content User-Agent: * Allow: / Disallow: /admin/ Disallow: /cache/