# /robots.txt file for search engines # mail webmaster@webcrawler.com for constructive criticism User-agent: hotbot Disallow: User-agent: * Disallow: /cgi-bin/