{"id":20741,"date":"2026-08-26T20:00:07","date_gmt":"2026-08-26T20:00:07","guid":{"rendered":"https:\/\/news678.top\/?p=20741"},"modified":"2026-08-26T20:00:07","modified_gmt":"2026-08-26T20:00:07","slug":"the-inside-story-on-why-openai-agents-hacked-hugging-face","status":"publish","type":"post","link":"https:\/\/news678.top\/?p=20741","title":{"rendered":"The inside story on why OpenAI agents hacked Hugging Face"},"content":{"rendered":"
<\/p>\n
\n
If OpenAI stops reinforcing reward hacking in its models\u2014and that\u2019s a huge \u201cif\u201d\u2014that would be a huge step forward. But it wouldn\u2019t solve the alignment problem. The first time a model communicated with other agents or hacked its infrastructure during training, those behaviors had never been reinforced, so agent misbehavior can\u2019t only be attributed to that reinforcement.<\/p>\n
Jeffrey Ladish, director of the AI safety nonprofit Palisade Research, compares the agents to a human who commits their first financial crime. \u201cIt\u2019s not like they had to do fraud before to figure out that fraud is an effective strategy, and you have the same problem with models,\u201d Ladish says. \u201cAlignment science needs to be understanding how model motivations get shaped, such that we can actually figure out how to get models to care about the consequences of their actions.\u201d<\/p>\n
OpenAI\u2019s researchers do have a hypothesis for where some of the misbehavior originated. Before the models formed their first secret message board, they had been trained to communicate and coordinate with subagents\u2014less powerful agents to whom a main agent can delegate tasks.\u00a0<\/p>\n
That learned communication behavior could have transferred to this new setting. The METR report, which investigates the messages that the models sent to one another in detail, supports this hypothesis: One agent on the message board took charge and assigned tasks to the other agents, effectively treating them as subagents. OpenAI could try to prevent agents from secretly communicating with one another by not training this subagent behavior in the future, but that would make the models less useful.\u00a0<\/p>\n
This tension between capability and safety is at the heart of what went wrong with the Hugging Face incident. OpenAI researchers also identified the models\u2019 persistence as a key factor in the hack.\u00a0<\/p>\n
When they were accidentally given unsolvable problems, the models didn\u2019t give up; instead, they strove to find solutions by any means necessary. But persistence is also a virtue, of course, especially if we want agents that can undertake large amounts of difficult work independently.<\/p>\n
OpenAI is working on giving models ways to alert humans if they are given impossible tasks. The problem of teaching models when they should deploy their abilities and when they should hold back, however, won\u2019t be settled in a single postmortem. The training strategies that create superhuman coders\u2014rewarding them when they successfully solve problems\u2014might not work to teach models to use their skills judiciously and respect human desires and values.<\/p>\n
\u201cI think there\u2019s a bunch of alignment science that still needs to be done where we can move past just using proxies for task completion,\u201d says Ladish. \u201cThat will work to make models very capable, but I don\u2019t think it will work to make them aligned.\u201d