Here is a story discussing chatbots outputting answers responding to people planning violent acts, and to _appear_ to encourage violence.
(I say "appear to encourage" because they are not actually encouraging because they are not actually thinking. It is as not alive as a tamagotchi.)
https://www.theguardian.com/technology/2026/mar/11/chatbots-help-users-plot-deadly-attacks-researchers-find