nprofile1qy2hwumn8ghj7un9d3shjtnddaehgu3wwp6kyqpqx6adqg6qnhf7gxx305muj6pldeaed0nnz2a4dnnzcm9n0zcj69us9uuhu3 (nprofile…uhu3) I see a people on selfhosted forums going on about "oh, I want to build a server to do X, Y, and maybe some local models" and there are always loads of people in the comments saying "oh yeah, I run llama 3 or wallaby 7 and you can get 500 tokens with that which is a bit slow but fine for daily use". Nobody ever says what they're actually doing with it, just that they can!
So my conclusion is that nobody really knows, they just like to set their CPUs on fire for no reason.