# /robots.txt file for http://www.brightporn.com # Questions to web@brightporn.com User-agent: * Disallow: /cgi-bin/ Disallow: /server/ Disallow: /work/ Disallow: /test/ Disallow: /tmp/ Disallow: /go/ Disallow: /setup.sgc Disallow: /robot.sgc User-agent: linksmanager_bot Disallow: / User-agent: EmailCollector Disallow: / # There is no illegal content on our sites. Please protect your rights for your own money. Do not spider us or pay our traffic bills. User-agent: NPBot Disallow: / # No stress tests here user-agent: stress-agent Disallow: / # Use browser to viewing our site User-agent: SiteSnagger Disallow: / User-agent: WebStripper Disallow: / User-agent: WebCopier Disallow: / User-agent: Fetch Disallow: / User-agent: Offline Explorer Disallow: / User-agent: Teleport Disallow: / User-agent: TeleportPro Disallow: / User-agent: WebZIP Disallow: / User-agent: linko Disallow: / User-agent: Xenu Disallow: / User-agent: larbin Disallow: / User-agent: ZyBORG Disallow: / User-agent: Download Ninja Disallow: / User-agent: wget Disallow: / # MAD Robots User-Agent: MJ12bot Disallow: / User-Agent: my-heritrix-crawler Disallow: /