Troed Sångberg on Nostr: Here's the reader's digest: You claimed that the only reason the 12B parameter LLM I ...

Here's the reader's digest:

You claimed that the only reason the 12B parameter LLM I used (that's 6.3 GB of data) could interpret the image the same way as Bender was because (and I quote) "similar couple image + description has been part of the training dataset". When I said that your claim is provably wrong you reiterated that I would have to show that "there was no sample in the "training" data of Claude associating a similar post-it near a doorbell to a similar description."

To support your claim that this 6.3 GB file on my hard drive somehow has similar images and descriptions to base its interpretation from you the linked a paper showing that LLMs that overfit/finetune can achieve comparable compression ratios to PNG, ZIP etc. On images, around %50 compression ratio.

So. You claim that 6.3 GB somehow incudes lossily compressed versions of all relevant images os post-its, doorbells and descriptions that exist on society. This is laughable by itself - even BEFORE we recall that for your claim to be generalized it must also hold true for ALL OTHER possible images this thread could've been true.

That fails the "not in any universe possible by know physical laws" test.

I'm guessing you have an opinion on LLMs that no facts in the world will ever change. And that's ok. It's just wrong.

LLMs have, ever since they started to be trained on large enough data sets, moved on from storing representations of training data to instead store higher level concepts. Today's LLMs are no longer "language models" as much as they are "thinking machines" (which also is why Bender's credentials aren't very relevant) and they reason at higher abstraction levels - fully capable of understanding the concepts of hand written notes, doorbells and possible scenarios that could have happened without needing similar images in the training data.