Wulfy on Nostr: I agree, this kind of oversight is atypical. I found it quite hard to get accurate ...

I agree, this kind of oversight is atypical.

I found it quite hard to get accurate consumption data out of the models.

Token caching is most certainly baked in the Models. So ordinary users will still experience ti.-The reason for it is why early internet providers invested heavily in cache systems, to save $.

At the end of the day, I strongly suspect that the Ai mongers are charching as much as the market will bear.

It has less to do with accurate accounting per token, but extracting maximum value that the client is prepared to pay for, hence the token accounting is very elastic.