I also went with Yoto with all the reservations you have. But it is really quite good hardware. While all the software is proprietary, inside is an ESP-32 and I am counting on nerd parents like us to eventually flash libre yoto to it backed with community run digital infra.
You will find there is quite a lively scene of people doing unofficial mods, teardowns, shadow audio book exchanges etc which made me confident the hardware will not go fully obsolete in the future. In the mean time however, I can recommend it. Due to language I don't even use the official cards, but I bought a stack of empty ones and we use that to make cards in our own languages. You should also note the cards can also be linked to an RSS feed so you can either use public radio podcasts or distribute your own audio as podcast ;)