Why don't the *&^(*& people at Meta / Facebook honour my robots.txt? They are crawling my linked data dereferencer like crazy.
I had to block them using a firewall
#meta #scraping #ethics #webethics
René Voorburg
npub12u…550c8
2026-05-31 11:38:38 UTC
Author Public Key
npub12u9kqctlz0k3n94kjact5phh0022jfndz0g0f4slu2azdqzmk8eqm550c8Seen on
wss://relay.momostr.pinkPublished at
2026-05-31 11:38:38 UTCEvent JSON
{
"id": "4f105cc348063657593b5ba8437efbaae064391f54b34fa50d58f9590822befe",
"pubkey": "570b60617f13ed1996b69770ba06f77bd4a9266d13d0f4d61fe2ba26805bb1f2",
"created_at": 1780227518,
"kind": 1,
"tags": [
[
"t",
"ethics"
],
[
"t",
"scraping"
],
[
"t",
"meta"
],
[
"t",
"webethics"
],
[
"proxy",
"https://code4lib.social/@digitopia/116668990623309786",
"web"
],
[
"proxy",
"https://code4lib.social/users/digitopia/statuses/116668990623309786",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://code4lib.social/users/digitopia/statuses/116668990623309786",
"pink.momostr"
],
[
"-"
]
],
"content": "Why don't the *\u0026^(*\u0026 people at Meta / Facebook honour my robots.txt? They are crawling my linked data dereferencer like crazy.\n\nI had to block them using a firewall\n\n#meta #scraping #ethics #webethics",
"sig": "90bcfd3dde8b84326724a711ad5903172e5769b44d1fe29b07175c50beee50505b70d53480e8d7cbb65e300f03e2adf8bea0d6c00236f23a6dab9246f54f96f8"
}