benjojo on Nostr: equinox ROCm is just a bit of a ongoing disaster, depending on what you're doing, you ...

equinox (nprofile…cfp5) ROCm is just a bit of a ongoing disaster, depending on what you're doing, you may find that anything with a Vulcan back end will be far more pleasant than trying to make any Room thing work well, also yes the 'tiny' models like that 12Bil one are not really useful for general tasks, potentially okay for things like spam filtering or fixed tasks that don't need much '''thinking''' But I wouldn't try anything fancy on them. Modern models people are using to code/chat with via outsourced compute are likely somewhere in the 1+ trillion parameter class