User-agent: * Crawl-delay: 5 Disallow: /Login.aspx Disallow: /PasswordRecovery.aspx Disallow: /xyz.ashx