alina (enterprise edition) on Nostr: oh cool great my dataset failed to import at 31 million out of 38 million matching ...
oh cool great my dataset failed to import at 31 million out of 38 million matching candidates, after searching through 8TB and indexing 250GB of metainfo into postgres, and now i can basically start anew - not because it would be hard to recover the lost progress but because i entirely overengineered the shit out of this deployment and it automatically scales up to hundreds of 16 core 32gb ram cloud servers because i got tired of being ratelimited when writing shell scripts to wrap around opensearch-cli so i wrote a pure rust+nix+nushell "deployment controller" to speed things up lmao
Published at
2026-10-07 23:09:18 UTCEvent JSON
{
"id": "ff49a125129dfddfabd6d27309b7f261a237e03f21d82ed9f0994a18c095549a",
"pubkey": "d5ec47da14e2552b1757fd4fcf94f090ca749a35073341144e21d986527bfa86",
"created_at": 1791414558,
"kind": 1,
"tags": [
[
"proxy",
"https://girldick.gay/@alina/117402144497444466",
"web"
],
[
"proxy",
"https://girldick.gay/users/alina/statuses/117402144497444466",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://girldick.gay/users/alina/statuses/117402144497444466",
"pink.momostr"
],
[
"-"
]
],
"content": "oh cool great my dataset failed to import at 31 million out of 38 million matching candidates, after searching through 8TB and indexing 250GB of metainfo into postgres, and now i can basically start anew - not because it would be hard to recover the lost progress but because i entirely overengineered the shit out of this deployment and it automatically scales up to hundreds of 16 core 32gb ram cloud servers because i got tired of being ratelimited when writing shell scripts to wrap around opensearch-cli so i wrote a pure rust+nix+nushell \"deployment controller\" to speed things up lmao",
"sig": "096178fa3280f40ac0f7ff709fd5f3daadb1d01c88b0a7b51d1f5b972a4e2b12008fd71fec1077c7ac74ff327075d311b6c62ed31fd4ca6f165d2bdaf8457c76"
}
