User-agent: Amazonbot # Amazon's user agent Disallow: / User-agent: NPBot Disallow: /do-not-crawl/ User-agent: Yandex Disallow: / User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: SeznamBot Disallow: / User-agent: DotBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: AspiegelBot Disallow: / User-agent: PetalBot Disallow: / User-agent: Seekport Disallow: / User-agent: amazonaws Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Bytespider Disallow: / User-agent: AhrefsBot Disallow: / User-agent: bingbot Disallow: / User-agent: gptbot Disallow: / User-agent: AwarioBot Disallow: / User-agent: DataForSEO Disallow: / User-Agent: * Crawl-Delay: 3600