User-agent: * Disallow: /cache/ User-agent: Yandex Disallow: / User-agent: AhrefsBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: coccocbot-web Disallow: / User-agent: PetalBot Disallow: / User-agent: GPTBot Disallow: / User-agent: CensysInspect Disallow: / User-agent: zgrab Disallow: / User-agent: Scrapy Disallow: / User-agent: Bytespider Disallow: /