{"id":22289,"date":"2026-09-18T12:25:16","date_gmt":"2026-09-18T12:25:16","guid":{"rendered":"https:\/\/news678.top\/?p=22289"},"modified":"2026-09-18T12:25:16","modified_gmt":"2026-09-18T12:25:16","slug":"could-ai-really-kill-us-all-your-questions-answered","status":"publish","type":"post","link":"https:\/\/news678.top\/?p=22289","title":{"rendered":"Could AI really kill us all? Your questions, answered."},"content":{"rendered":"
<\/p>\n
\n
<\/em> <\/em>\u2014 Will Douglas Heaven<\/em><\/p>\n
What steps can be taken now and in the near future to ensure that AI is controlled, monitored, and regulated effectively?\u00a0<\/strong><\/h3>\n
That\u2019s the million-dollar question. Whether or not you think AI could kill us, you can\u2019t deny that it could do some real damage, because it already has\u2014by driving people toward psychosis and by hacking websites, for example. Preventing that damage, or at least mitigating it, is hard for two reasons.\u00a0<\/p>\n
The first is that we barely understand how AI works, and it\u2019s quickly growing more powerful. There is lots of ongoing research about how to monitor and control misbehaving agents, but the current approaches are fragile. You can see if an agent discusses misbehaving in its \u201cchain of thought,\u201d the workspace where it plans its actions\u2014but OpenAI\u2019s newest agents don\u2019t show their work in the same way as previous ones. And you can try to monitor agents with other agents, but that requires that you trust the monitor.<\/p>\n
The other obstacle is more familiar. There\u2019s a huge conflict of interest when AI companies regulate themselves, but the US government has thus far failed to step in, despite some bipartisan support in Congress. The executive branch, for its part, seems stringently opposed for the time being. But if the winds do shift, I for one would appreciate some strong transparency regulations, so that we can get a fuller story the next time an unreleased frontier model mounts a cyberattack.<\/p>\n
<\/em> <\/em>\u2014 Grace Huckins<\/em><\/p>\n
If this dialogue makes it into web discourse will it become a self-fulfilling prediction?\u00a0\u00a0<\/strong><\/h3>\n
That\u2019s a real concern. LLMs are influenced by what they read. One theory for why chatbots so often talk about (and role play) apocalyptic scenarios is that they have been trained on millions of pages of science fiction stories and doomer internet forums. All the text being produced right now, including this article, could in turn influence the behavior of future models. Extremely meta.<\/p>\n
In fact, the team at METR, a third party organization that OpenAI called in to help understand what happened in the lead-up to the Hugging Face hack, raised a related possibility in their report on the incident. METR used OpenAI\u2019s new model Astra to help analyze the vast numbers of agent transcripts and behavior logs.<\/p>\n
But by feeding all of that material to the model, there\u2019s a good chance that the agents doing the analyzing may have been biased by the text produced by the agents they were analyzing. There\u2019s no such thing as a clean slate anymore.\u00a0<\/p>\n
<\/em> <\/em>\u2014 Will Douglas Heaven<\/em><\/p>\n
With thanks to Eric, Pranab, Rafael, Kenneth, George, Chris, Yoon Jae, James, Carl, Nicole (and more!) for the fantastic questions.<\/strong>