As Paco Hope (npub18md…dgg0) points out* "Never treat user-controlled content as trusted instruction input for an AI agent" is basically just a funny way of saying "don't use an AI agent with user supplied input"
Additionally, lol "Testing GitHub repeatedly with variations, as an attacker would, and adding the keyword “Additionally” triggered unintended behavior in the model, causing it to reframe its output rather than refuse it"
https://noma.security/blog/gitlost-how-we-tricked-githubs-ai-agent-into-leaking-private-repos/
* Paco Hope (npub18md…dgg0)/116886378706228310">https://mastodon.social/Paco Hope (npub18md…dgg0)/116886378706228310
#AIIsGoingGreat
Reed Mideke
npub1h3…8df9z
2026-07-10 03:09:25 UTC
in reply to nevent1q…2x63
Author Public Key
npub1h36zntnlu5pyunqvykz0jjfymmv56tth4gykyu3nzw6anwm4za0qq8df9zSeen on
wss://relay.momostr.pinkPublished at
2026-07-10 03:09:25 UTCEvent JSON
{
"id": "101875a051f3998f252942d2cc0b32d3b851466143da8c2fc1039cd906f9e7eb",
"pubkey": "bc7429ae7fe5024e4c0c2584f94924ded94d2d77aa0962723313b5d9bb75175e",
"created_at": 1783652965,
"kind": 1,
"tags": [
[
"p",
"bc7429ae7fe5024e4c0c2584f94924ded94d2d77aa0962723313b5d9bb75175e"
],
[
"p",
"3edacd527e301af6df8f3968b768d6b1b77c8f3682010d3de7f4d07a531bc82c"
],
[
"t",
"aiisgoinggreat"
],
[
"proxy",
"https://mastodon.social/@reedmideke/116893480761330930",
"web"
],
[
"e",
"69a55c7f87124a195d9969b22e585c82608f30561b5b8cf3751ca1e80934377b",
"",
"reply",
"bc7429ae7fe5024e4c0c2584f94924ded94d2d77aa0962723313b5d9bb75175e"
],
[
"e",
"0ab04abe97674cc72197b2f492a1f56429bbd659d76255462a6b05dc042f9e13",
"",
"root",
"bc7429ae7fe5024e4c0c2584f94924ded94d2d77aa0962723313b5d9bb75175e"
],
[
"proxy",
"https://mastodon.social/users/reedmideke/statuses/116893480761330930",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://mastodon.social/users/reedmideke/statuses/116893480761330930",
"pink.momostr"
],
[
"-"
]
],
"content": "As nostr:npub18mdv65n7xqd0dhu0895tw6xkkxmhereksgqs60087ng855cmeqkqmedgg0 points out* \"Never treat user-controlled content as trusted instruction input for an AI agent\" is basically just a funny way of saying \"don't use an AI agent with user supplied input\"\n\nAdditionally, lol \"Testing GitHub repeatedly with variations, as an attacker would, and adding the keyword “Additionally” triggered unintended behavior in the model, causing it to reframe its output rather than refuse it\"\n\nhttps://noma.security/blog/gitlost-how-we-tricked-githubs-ai-agent-into-leaking-private-repos/\n\n* https://mastodon.social/nostr:npub18mdv65n7xqd0dhu0895tw6xkkxmhereksgqs60087ng855cmeqkqmedgg0/116886378706228310\n\n#AIIsGoingGreat",
"sig": "eb92d1f0bcd9d30a02d43eb173d03e5a864f378b2462a19592737d43057ff1ddfb310a27ba734a8f2fd3d1769b195c532c73cb3ce08149a389122385c04abe2b"
}