Peter Bloem

Assistant prof. at the Learning and Reasoning group, Vrije Universiteit Amsterdam.

Not an AI pusher, I promise. I just think we need a better class of criticism.

website: https://peterbloem.nl
group: https://lr.cs.vu.nl
pronouns: he/him

Last Notes

2026-10-26 21:37:21 UTC

- reply

Social impact statement: This technology allows population-level massacre at a cost of a hundred dollars per person. Produced at scale it could cause more suffering and death per dollar than nuclear weapons.
However, there are also positive uses: it could be used to help runners improve their form.

2026-10-26 21:32:55 UTC

Autonomous drones following running people through the woods, without GPS. Developed, with military funds, at a public university.
This is not dual use anymore, this is single use. There is only one use here.
Sometimes I worry that I'm becoming a negative Nancy with all my ethics talk, and then I see colleagues proudly post this kind of thing without a single acknowledgment of the implications.
https://media.mathstodon.xyz/media_attachments/files/117/509/331/336/294/921/original/09693867991c287a.png

2026-10-25 11:15:25 UTC

- reply

Arguably, something like the Iran war already qualifies, since Claude seems to have played a role in the decision to go to war, but we're looking for a more direct involvement here. Something that will make the general public say that "AI was responsible for this."

2026-10-25 11:13:57 UTC

How long until AI unintentionally*, directly causes a single incident in which (a) at least hundreds of people die or (b) at least tens of thousands of people suffer something of comparable magnitude to losing your life's savings?
* Not intended by any human being. It may be that the AI itself has some vague notion of intention or that it just craps out in some random way.

2026-08-26 17:06:34 UTC

- reply

I'll try to keep it brief. :)
It's true that offline statistical analysis can only uncover correlation not causation, so that's a limit to what a purely autoregressive LLM can do.
There are two important counters though:
- LLMs aren't trained just autoregressively. They are also trained by RL, which does admit causal analysis to emerge. Admittedly, most of the power comes from the pre-training, but this removes the impossibility argument.
- A huge amount of behaviors don't require understanding of the causality of the training data. Including, and this is I admit is counter-intuitive, running causal analysis in-context. An LLM could learn to run programs from a suffiently large dataset, and those programs could include existing causal analysis algorithms, which it could then apply in context to data.
- Modern models also have a kind of internal monologue. This pushes its behavior quite far from "sampling from the data distribution", especially coupled with RL.

2026-08-26 13:03:18 UTC

- reply

I see, that makes sense. I agree that that in that sense there are fundamental parts missing.
I don't think they require deep conceptual breakthroughs to achieve, but that's then not relevant to the discussion (and other AI people do believe there are big breakthroughs required, so I'm not claiming to have the final answer here).

2026-08-26 12:39:24 UTC

- reply

At this point I'm more weary of old AI luminaries claiming there is a big piece of the puzzle missing (and they have the answer) than I am of big AI labs. LeCun, Sutskever and all the others have a huge incentive to make blind claims like these to raise funds and reclaim relevance. I'm not seeing much evidence in these discussions.

2026-08-26 09:24:01 UTC

- reply

> But for AI to be better at math than humans are, it would need to be fundamentally different than current LLMs.
What makes you say that? I agree that there are some gaps to general intelligence (persistence, agency, identity, long-term memory, a stake in the world) but most of those seem engineering problems rather than anything fundamental.
The hard problems (intuition, common sense, natural language use, tool use, reflection) all seem to have been solved.
What do you see as the fundamental parts that are still missing?

2026-08-21 14:26:28 UTC

- reply

I know. It's not what I'm saying or thinking though.

2026-08-21 12:41:29 UTC

- reply

Let me go point by point here.
- Curation doesn't mean manual selection. It just means that you don't blindly ingest everything from the internet. This has been done since GPT2, when they used social media upvotes as a proxy for article quality. In GPT3, they included sources proportional to the quality of the source which gives you the best of both worlds.
- All sorts of junk sneaks into the data, but so long as the majority is high quality, it won't lead to model collapse.
- "Glue on Pizza" came from the Google AI summary which is a poor proxy for current AI capability.
- Watermarking can and does 100% work as advertised. For a simple insight to the idea, consider the bitstream used to feed the PRNG. If you have the complete model output of one session and the model, you can reconstruct that. Save it, use the same stream each session and you have your watermark with no changes to the model (some extra tricks are necessary to watermark subsets of the conversation).
- It is indeed relatively easy to circumvent, although the precise method they use seems to be somewhat robust to basic rephrasing. Still, it's going to catch out a lot of lazy people.

2026-08-21 11:59:05 UTC

- reply

My understanding is that it's mostly happening because the EU requires it.
Pre-training datasets are pretty carefully curated and they have strong quality filters, so I don't think there is much of a risk of model collapse in practice.
It's also not so much about special characters or word choice as about the choices between equally good options (by the models' own probabilities). Every time you choose between two equally likely options you are essentially "encoding" one bit (i.e. a coinflip) of information. You can use these bits to create a watermark without losing quality.

2026-08-16 10:05:53 UTC

🥖 A new blog post! 🥖 (Yes another one.)
The question of AI coding is usually seen as a binary choice. You buy in or you don't. It's not, and it's important that we move the discussion on from that.
Vibe-coding is terrible. I creates sloppy code in drifting, unmaintained code bases, it rapidly deskills you and it wastes resources. But vibe-coding isn't the only fruit.
https://peterbloem.nl/blog/craft-coding

2026-08-14 20:45:37 UTC

https://op.tngl.io/every-fucking-website/

2026-08-13 11:44:05 UTC

- reply

From https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD898.html
(This is the source of the famous "Can Submarines Swim" quote.)

2026-08-13 11:37:58 UTC

Just found this quite chilling line in a Dijkstra lecture from 1984.
https://media.mathstodon.xyz/media_attachments/files/117/087/994/355/582/216/original/d5c9d61d3a7559c8.png

2026-07-25 11:04:43 UTC

- reply

I would say on a higher level ML models are good for modeling "an intuition" about interesting directions. This is helpful in automated proof search (or game playing) as a fast heuristic to guide the search.
For human/AI interaction, the question then becomes whether "an intuition" is "a human-like intuition". There, I think the models have a tendency to diverge from human instincts, which makes it difficult to collaborate. They "experience" the world very differently to us and what is simple to them can be very complex to us.

2026-07-24 09:57:39 UTC

- reply

I'm somewhat hopeful that mathematics will become more about understanding than about who can prove the most theorems.
Once the frontier of what can be proven is entirely the domain of the machines, the question will be: what part of it can we still understand?
That kind of mathematics would require a whole new set of skills, and tools, but I think it's a worthwhile pursuit.