Rob Ricci on Nostr: There is another kind of defense that LLMs can potentially be useful for that I ...

There is another kind of defense that LLMs can potentially be useful for that I skipped over before. Largely, because my thinking on it is less developed at this point. Yes, LLMs can help us find - and patch - security vulnerabilities. But what about the "LLMs as cops" or "good guy with an LLM" argument, where they are used to provide active defense against the threats from other LLM-discovered vulnerabilities or agent-driven attacks?

This sounds like a dystopia to me. What's the fundamental difference between a cop-agent taking action against a "bad guy" agent, and a cop-agent taking (digital! I'm not talking about RoboCop here - I hope!) action against a person on the network? How can it even know? In the physical world, we have processes for allocating resources to, and limiting, policing. I'm not going to call these processes "good." In the US, for instance, I think we have gone much too far in militarizing our police (along with many other ways we do policing wrong). But now imagine the agentic version of this. SWAT teams, at scale? Whoever has the biggest datacenter gets to enforce their own version of "safety"? It's hard to see how this ends up anywhere other than extreme digital authoritarianism.

This does not rule out LLMs for defense, but positions them as tools for defenders, not agents with a cop badge and a gun.