Jazzilla on Nostr: Thanks for the new quant. I still use unsloth and bartowski quants. They are bigger, ...

Thanks for the new quant.

I still use unsloth and bartowski quants. They are bigger, but very high quality.

My unsloth q4 k m routinely outhinks the latest Claude.

For really hard problems, I always break it out to see what it says. It hasn't let me down yet.

I turned the corner on this local ai stuff when I learned you really just need a large context (10 gig), and running in 30 gig of vram all in is fine.

Plenty of left over memory with arm soc uma.