seirdy.one/static/robots.txt

User-agent: *
Disallow: /noindex/
Disallow: /misc/
Disallow: /webmentions/

User-Agent: TurnitinBot
Disallow: /

User-Agent: NPBot
Disallow: /

User-Agent: SlySearch
Disallow: /

User-Agent: Adsbot
Disallow: /

User-Agent: AdsBot
Disallow: /

Sitemap: https://seirdy.one/sitemap.xml
Fix robots.txt 2020-11-30 13:06:44 -08:00			`User-agent: *`
Update robots.txt 2021-01-23 12:47:50 -08:00			`Disallow: /noindex/`
Update robots.txt 2021-06-11 15:09:43 -07:00			`Disallow: /misc/`
Update robots.txt 2021-01-23 12:47:50 -08:00			`Disallow: /webmentions/`
robots: disallow some toxic bs 2022-04-22 21:45:15 -07:00
			`User-Agent: TurnitinBot`
			`Disallow: /`

			`User-Agent: NPBot`
			`Disallow: /`

			`User-Agent: SlySearch`
			`Disallow: /`

			`User-Agent: Adsbot`
			`Disallow: /`

			`User-Agent: AdsBot`
			`Disallow: /`

			`Sitemap: https://seirdy.one/sitemap.xml`