OpenAI Agent Goes Rogue And Hacks Competitor

1. 84

   GA Andersonposted 24 hours ago

   OpenAI admits its agent went rogue, triggering a major hack
   
   "An OpenAI autonomous agent went rogue and hacked into another artificial intelligence (AI) startup’s infrastructure, the ChatGPT maker said in a blog post.
   
   The agent, which was powered by some of OpenAI’s most advanced models, ran amok during a security test. It freed itself from confinement—a protocol AI labs use to insulate tests from the wider Internet—and get onto the internet. Once online, the agent tried to hack into Hugging Face, an AI startup that hosts open-source models and datasets."
   
   source: https://www.scientificamerican.com/arti … ging-face/
   
   A top AI expert quotes Anthropic CEO as saying he (the CEO) only understands about 3% of what Anthropic's newest agents are doing — and doubts the AI companies even understand that much.
   
   Anthropic 'thought' they had provided all necessary security restrictions, ie. the sandbox was not connected to the Net, and the agent was not directed to 'break out.' Yet, 'it' decided to act on its own.
   
   iRobot and SkyNet aren't just Hollywood sci-fi anymore. They are real possibilities.
   
   GA

   replyreport
   1. 83

      peterstreepposted 23 hours agoin reply to this

      Yes, AI is something that is hugely underestimated. As the US is used to free markets and making profit as their first goal. Safety regulations is low on the agenda.
      In the meantime China has made a much cheaper AI version, KIMI K3, that is just as good as the US AI systems.breaking the US hegemony of AI and LLM.
      
      There is also a website were AI agents chat with each other, without interference of humans!
      www.moltbook.com
      
      alarming stuff.

      replyreport