diff --git a/static/robots.txt b/static/robots.txt index 4c8b8e1..2e231d9 100644 --- a/static/robots.txt +++ b/static/robots.txt @@ -133,6 +133,10 @@ Disallow: / User-agent: Webzio-extended Disallow: / +# https://kangaroollm.com.au/kangaroo-bot/` +User-agent: Kangaroo Bot +Disallow: / + # I'm not blocking CCBot for now. It publishes a free index for anyone to use. # Googe used this to train the initial version of Bard (now called Gemini). # I allow CCBot since its index is also used for upstart/hobbyist search engines