Dillo browser on Nostr: Moved our git web server from cgit to stagit to reduce load on the server caused by ...
Moved our git web server from cgit to stagit to reduce load on the server caused by crawlers, as it only uses static files.
https://git.dillo-browser.org/dillo/Roughly 99.9% of requests are bots fetching commits and files over and over again at a rate of 50 req/s despite our robots.txt excluding any crawler. Stagit only exposes a small number of files and commits so they are not reachable to bots.
Sadly, many anti-bot protections rely on JS or fingerprinting which come with their own set of problems.
Published at
2026-10-18 12:39:05 UTCEvent JSON
{
"id": "ac45037554533e787868036cbbe6ec05325b5be26c7fdd70787f9e87586cff73",
"pubkey": "67d7c8e41886705b55a238b120612083836dcd493c8f400bf45057db5651739f",
"created_at": 1792327145,
"kind": 1,
"tags": [
[
"proxy",
"https://fosstodon.org/@dillo/117461951777413540",
"web"
],
[
"imeta",
"url https://cdn.fosstodon.org/media_attachments/files/117/461/878/978/456/861/original/592142c4f1fa8bde.png",
"m image/png"
],
[
"proxy",
"https://fosstodon.org/users/dillo/statuses/117461951777413540",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://fosstodon.org/users/dillo/statuses/117461951777413540",
"pink.momostr"
],
[
"-"
]
],
"content": "Moved our git web server from cgit to stagit to reduce load on the server caused by crawlers, as it only uses static files.\n\nhttps://git.dillo-browser.org/dillo/\n\nRoughly 99.9% of requests are bots fetching commits and files over and over again at a rate of 50 req/s despite our robots.txt excluding any crawler. Stagit only exposes a small number of files and commits so they are not reachable to bots.\n\nSadly, many anti-bot protections rely on JS or fingerprinting which come with their own set of problems.\nhttps://cdn.fosstodon.org/media_attachments/files/117/461/878/978/456/861/original/592142c4f1fa8bde.png\n",
"sig": "efe4415aab77475fcb092d81a44aad3313ca4d735a9a806f5ee5db24c72b99bc2ebeb1e874db1ead804b52823827e15897dbed355562825596543ab7399ec466"
}
