Of the many crawlers, the 3 most misbehaved - so far as basics like honouring HTTP 429 - are in order:
meta-externalagent/1.1
Applebot/0.1
Amazonbot/0.1
Julian Oliver
npub1rc…8nu7f
2026-06-02 01:48:15 UTC
in reply to nevent1q…ajsr
Author Public Key
npub1rc8yx99dyrz0rqd0y5zu50hgwqsg0w0xqklg9l4xtej8yajyy47s68nu7fSeen on
wss://relay.momostr.pinkPublished at
2026-06-02 01:48:15 UTCEvent JSON
{
"id": "badc77d88e2e32829d5c802a04e7103986f84511ad0abda60cfc747e974074fd",
"pubkey": "1e0e4314ad20c4f181af2505ca3ee8702087b9e605be82fea65e64727644257d",
"created_at": 1780364895,
"kind": 1,
"tags": [
[
"p",
"1e0e4314ad20c4f181af2505ca3ee8702087b9e605be82fea65e64727644257d"
],
[
"proxy",
"https://mastodon.social/@JulianOliver/116677993768839884",
"web"
],
[
"e",
"aeefc6e303af705802b7364e3c02783770329320617ea60586cdc50d73b5e85a",
"",
"root",
"1e0e4314ad20c4f181af2505ca3ee8702087b9e605be82fea65e64727644257d"
],
[
"p",
"db9131f37cc0877131b99b0773aa0b41b20f72a409e5d46154a59a2ac726d9d5"
],
[
"e",
"4cb944e7714eba441380fccc0cc682dc977dbe27f7261a945901b6bb2a8844a4",
"",
"reply",
"1e0e4314ad20c4f181af2505ca3ee8702087b9e605be82fea65e64727644257d"
],
[
"proxy",
"https://mastodon.social/users/JulianOliver/statuses/116677993768839884",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://mastodon.social/users/JulianOliver/statuses/116677993768839884",
"pink.momostr"
],
[
"-"
]
],
"content": "Of the many crawlers, the 3 most misbehaved - so far as basics like honouring HTTP 429 - are in order:\n\nmeta-externalagent/1.1\nApplebot/0.1\nAmazonbot/0.1",
"sig": "8b87366e8115f67f514e4e24944e2509ac8401429754812be86bc259375d00e9c259b7c7e8e2854c8ddaa3f5fc17a3652a932d6005e0dfde4eafcbcdb53b9cd0"
}