Reed Mideke on Nostr: Newsflash! In-band signalling is still insecure: "LLMs identify roles from an ...

Newsflash! In-band signalling is still insecure: "LLMs identify roles from an insecure feature (style). This is like identifying a stranger's profession from how they talk and dress rather than by checking their ID. Usually everything agrees, so this works fine. But when attackers intentionally create a mismatch, the LLM uses the insecure method (writing style) to identify its role instead of the secure method (tags)"

https://role-confusion.github.io/

#AIIsGoingGreat