this is a reminder that LLMs hallucinate by default and guess correctly by luck. this is also a reminder that everything that an LLM tells you that can't be readily checked/evaluated should be treated like rubbish.
the fun thing is that there are plenty of use cases when you *can*, so they can be pretty useful! but don't ask it facts, or opinions, or choices.