From b731ec68d230852af204f7bc30620ace23f05a09 Mon Sep 17 00:00:00 2001 From: Seirdy Date: Tue, 10 Dec 2024 00:51:43 -0500 Subject: [PATCH] Update acknowledgement of jmjl to mention semrush --- content/meta/scrapers-i-block.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/meta/scrapers-i-block.md b/content/meta/scrapers-i-block.md index 917b3c4..eeedfc9 100644 --- a/content/meta/scrapers-i-block.md +++ b/content/meta/scrapers-i-block.md @@ -542,7 +542,7 @@ I found most bots via my server logs, but some third party collections proved he : I don't recall actually discovering a bot to block via Cloudflare Radar, but I'm linking it here as it's a helpful bot catalog in general. Be aware of errors in its bot categorization. Several bots listed under "AI Crawler" aren't actually training or even related to GenAI. Some "Search Engine Crawler" entries are non-crawling scrapers that only fetch favicons for search results. This is a good starting point for further research if you fact-check entries yourself. Helpful individuals -: Thanks to {{}} for pointing me out to the Open Web Search adding a `GenAI` product token for opting out of GenAI training. +: Thanks to {{}} for telling me about Open Web Search adding a `GenAI` product token for opting out of GenAI training, and about SemRush's array of product tokens.