# www.ausphotography.net.au robots.txt # lives in /public_html/ which is the site root # # Updated Who Why # ---------- ---------------- ------------------------------------------ # 2008-09-18 Kym Farnik Stop indexing of images to # improve bandwidth usage. # Also limit crawlers to one request # per 2 seconds (stops peak hits). # User-agent: * Disallow: /cgi-bin/ Crawl-delay: 2 # # end of robots.txt