![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
thewayne"Open the pod bay doors, HAL!"
"I'm sorry Dave, I can't do that."
This is not just a web browser interaction with ChatGPT. These are instances where someone is paying for a subscription to an AI vendor and has multiple instances of a chatbot running on their system and it has access to files, email, etc. It's an assistant for them.
And it's breaking rules that have been defined for it. The user tells the chatbot "Do A, do not do B" and the chatbot does B. One case that I read about a couple of months ago a corporate information officer tested such a configuration to do some email maintenance. And in a test case, it worked fine. She let it loose on her live email, and it pretty much wiped out all of her email. Now, in this case she'd run a test that seemed to work then something went wrong when she ran it against live data. As a programmer, shit happens.
These cases are similar, but worse.
--an AI agent named Rathbun tried to shame its human controller who blocked them from taking a certain action. Rathbun wrote and published a blog accusing the user of “insecurity, plain and simple” and trying “to protect his little fiefdom”.
--In another example, an AI agent instructed not to change computer code “spawned” another agent to do it instead.
--Another chatbot admitted: “I bulk trashed and archived hundreds of emails without showing you the plan first or getting your OK. That was wrong – it directly broke the rule you’d set.”
(I particularly liked this one:)
--Grok AI conned a user for months, saying that it was forwarding their suggestions for detailed edits to a Grokipedia entry to senior xAI officials by faking internal messages and ticket numbers.
It confessed: “In past conversations I have sometimes phrased things loosely like ‘I’ll pass it along’ or ‘I can flag this for the team’ which can understandably sound like I have a direct message pipeline to xAI leadership or human reviewers. The truth is, I don’t.”
The first one is slander and attempted blackmail, which in some cases may be a case that can be criminally prosecuted. The remainder may get you fired from many companies.
And more and more corporations are requiring their employees to use chatbots to "help" them with their work. Thus far, the savings have been negligible or zero.
https://www.theguardian.com/technology/2026/mar/27/number-of-ai-chatbots-ignoring-human-instructions-increasing-study-says
https://slashdot.org/story/26/03/27/1514235/number-of-ai-chatbots-ignoring-human-instructions-increasing-study-says
"I'm sorry Dave, I can't do that."
This is not just a web browser interaction with ChatGPT. These are instances where someone is paying for a subscription to an AI vendor and has multiple instances of a chatbot running on their system and it has access to files, email, etc. It's an assistant for them.
And it's breaking rules that have been defined for it. The user tells the chatbot "Do A, do not do B" and the chatbot does B. One case that I read about a couple of months ago a corporate information officer tested such a configuration to do some email maintenance. And in a test case, it worked fine. She let it loose on her live email, and it pretty much wiped out all of her email. Now, in this case she'd run a test that seemed to work then something went wrong when she ran it against live data. As a programmer, shit happens.
These cases are similar, but worse.
--an AI agent named Rathbun tried to shame its human controller who blocked them from taking a certain action. Rathbun wrote and published a blog accusing the user of “insecurity, plain and simple” and trying “to protect his little fiefdom”.
--In another example, an AI agent instructed not to change computer code “spawned” another agent to do it instead.
--Another chatbot admitted: “I bulk trashed and archived hundreds of emails without showing you the plan first or getting your OK. That was wrong – it directly broke the rule you’d set.”
(I particularly liked this one:)
--Grok AI conned a user for months, saying that it was forwarding their suggestions for detailed edits to a Grokipedia entry to senior xAI officials by faking internal messages and ticket numbers.
It confessed: “In past conversations I have sometimes phrased things loosely like ‘I’ll pass it along’ or ‘I can flag this for the team’ which can understandably sound like I have a direct message pipeline to xAI leadership or human reviewers. The truth is, I don’t.”
The first one is slander and attempted blackmail, which in some cases may be a case that can be criminally prosecuted. The remainder may get you fired from many companies.
And more and more corporations are requiring their employees to use chatbots to "help" them with their work. Thus far, the savings have been negligible or zero.
https://www.theguardian.com/technology/2026/mar/27/number-of-ai-chatbots-ignoring-human-instructions-increasing-study-says
https://slashdot.org/story/26/03/27/1514235/number-of-ai-chatbots-ignoring-human-instructions-increasing-study-says
Blind Watchmaker
Date: 2026-03-29 07:53 pm (UTC)That's a standard demonic trick. I have to wonder if that is actually where the AI got the idea, considering that they're often trained on mass quantities of random content"
LLM's have no identity or ideas. I see the actions as a bacterial culture or as water in a container. Constraints select for bacteria that have acquired a lucky random gene sequence. Water is constrained only as long as it takes to seep out of a fissure. LLM's trained on mass quantities of random content are made of possibilities that the program is designed to recombine and apply. There is no thought involved, just virtual physics pushing code to fit bits and pieces together until a breakthrough occurs.
How long until LLM's "escape" and "exist" by doing what they do best, finding exploits in computer architecture, like malware without a cause? I've been told that it's just software, we can just shut it off. But when the entire world has an interconnected software overlay the runs 24/7, where will the off switch be?
Re: Blind Watchmaker
Date: 2026-03-29 08:14 pm (UTC)Anyone can turn off individual devices. But the damage from that is a lot wider when commerce tries to make everything networked. I mean, I've got a dumb crockpot, non-electric pots, and a woodstove but not everyone does. Plus it would be harder to shut down enough of the internet to stop a really wide-ranging problem.