From c02e2a78a5ccca73f103719c352fc44e5558b963 Mon Sep 17 00:00:00 2001 From: Rohan Kumar Date: Mon, 24 Jul 2023 23:43:13 -0700 Subject: [PATCH] Add another IP-violation crawler to robots.txt --- static/robots.txt | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/static/robots.txt b/static/robots.txt index f9e3759..9ddcdc3 100644 --- a/static/robots.txt +++ b/static/robots.txt @@ -38,6 +38,11 @@ Disallow: / User-agent: CheckMarkNetwork/1.0 (+https://www.checkmarknetwork.com/spider.html) Disallow: / +# Stop trademark violations and affiliate non-compliance in paid search. Automatically monitor your partner and affiliates’ online marketing to protect yourself from harmful brand violations and regulatory risks. We regularly crawl websites on behalf of our clients to ensure content compliance with brand and regulatory guidelines. (https://www.brandverity.com/why-is-brandverity-visiting-me) +# --> fuck off. +User-agent: BrandVerity/1.0 +Disallow: / + # Eat shit, OpenAI. User-agent: ChatGPT-User Disallow: /