CrashLogger on Nostr: alternatively, prevent scraping by putting all your website content in robots.txt as ...

alternatively, prevent scraping by putting all your website content in robots.txt as it's been proven virtually none of the big scrapers ever read it.