seirdy.one/static/robots.txt

User-agent: *
Disallow: /noindex/
Disallow: /misc/
Disallow: /webmentions/

User-Agent: TurnitinBot
Disallow: /

User-Agent: NPBot
Disallow: /

User-Agent: SlySearch
Disallow: /

User-Agent: Adsbot
Disallow: /

User-Agent: AdsBot
Disallow: /

Sitemap: https://seirdy.one/sitemap.xml
Fix robots.txt 2020-11-30 21:06:44 +00:00			`User-agent: *`
Update robots.txt 2021-01-23 20:47:50 +00:00			`Disallow: /noindex/`
Update robots.txt 2021-06-11 22:09:43 +00:00			`Disallow: /misc/`
Update robots.txt 2021-01-23 20:47:50 +00:00			`Disallow: /webmentions/`
robots: disallow some toxic bs 2022-04-23 04:45:15 +00:00
			`User-Agent: TurnitinBot`
			`Disallow: /`

			`User-Agent: NPBot`
			`Disallow: /`

			`User-Agent: SlySearch`
			`Disallow: /`

			`User-Agent: Adsbot`
			`Disallow: /`

			`User-Agent: AdsBot`
			`Disallow: /`

			`Sitemap: https://seirdy.one/sitemap.xml`