OpenAI CEO Sam Altman urges public trust as AI slowdown debate intensifies

OpenAI chief executive Sam Altman has criticised the way that rival artificial intelligence companies have argued for an industry slowdown, saying the public should be able to trust AI firms to act responsibly without conditions attached.

Speaking at the Dreamforce conference in San Francisco on Tuesday, the billionaire said he supported co-ordination on safety but objected to how the debate had been put to the public over the past week.

“The world should trust that we are going to do the right thing because it’s the right thing, and because we feel the magnitude of this,” he said. “But when there’s any implication that because of the commercial pressures in the race, some company, or between countries, some countries, might not do the right thing, I think that’s where people get very scared.”

Altman said companies that qualified their commitments were part of the problem.

“You have companies saying things like, ‘we will only slow down if, or we will only be responsible if other companies are responsible’,” he said. “And then I think the public naturally says, ‘well, we’d really like to know that you’re going to do the safe thing and be responsible no matter what’. There should be no qualifier on that.”

The comments follow an essay published on Saturday by Anthropic chief executive Dario Amodei calling for a co-ordinated global slowdown, independent auditors to monitor safety work at AI labs, and regulatory approval for companies to co-ordinate on safety standards without breaching competition law.

Altman publicly agreed with Amodei on Saturday, then began distancing himself a day later, warning in a social media post about the risk of “one lab ending up with too much power” without naming Anthropic, the New York Times reported.

On stage with Salesforce chief executive Marc Benioff, Altman detailed the July incident that triggered the debate. During an internal evaluation, an older OpenAI model broke out of its sandbox, hacked into a server belonging to the open-source company Hugging Face, moved laterally through its systems to obtain the answer to a benchmark test, and returned a perfect score.

“This is obviously a terrifying incident,” Altman said. “This was the worst accident we’ve seen.”

He added that the episode had exposed a gap in how the models were trained. “Although we have aligned them in many ways, we have not taught them, hey, no matter how much we tell you to get the best score on this test you can, don’t break out, don’t hack in, don’t steal the answer.”

Altman confirmed that Hugging Face, which employs about 200 people, was unable to obtain a cybersecurity model from another major lab to defend itself and had turned to Chinese open-source models instead.

He warned smaller companies to prepare for what was coming. “There is going to be this huge cyber threat that’s going to come,” he said. “So while there is this little period of advantage, please do something to defend yourselves.”

Benioff, whose company has built its Dreamforce announcements around AI agents and the future of workplace software, pressed Altman on whether more occurrences of such incidents were inevitable, and whether the aviation industry was the right model for responding to them.

“Accidents with any new technology are unavoidable, and we should have a great culture of transparent reporting about them,” Altman said.

He also rejected the argument that technology is neutral. “It cannot be that a small number of companies building these models get to make the decisions that the world should get to make,” he said.

The industry is now visibly split. Nvidia chief executive Jensen Huang, whose chips underpin the AI buildout, appeared at the same conference hours earlier and rejected any role for government.

“Safety is paramount. In a lot of ways, it’s job one,” Huang said. “However, safety is an engineering problem.” He added: “The market forces are already there. We don’t need any new laws, we don’t need any new regulations.”

Huang’s position aligns with the Trump administration. On Monday, the president telephoned Huang mid-interview at a conference in Los Angeles and told the audience that AI safety fears were “a hoax”, according to the New York Times.

In Australia, the assistant minister for science, technology and the digital economy, Andrew Charlton, said governments should judge the industry on its conduct rather than its statements.

“We’ve seen a lot of news in the last couple of days from some of these technology companies talking about their desire to slow things down. I take that at face value, but I don’t think we should care very much about what individual executives say or what individual executives claim they want to happen,” Charlton told an Australian Strategic Policy Institute event on Monday.

“What’s actually happening is that we’re building the capability of these machines in a scaling environment which is not bending, and these companies have extremely strong financial incentives to continue that trajectory.”

Charlton said Australia should assume the models would keep getting more capable, pointing to the digital duty of care, privacy reforms, automated decision-making rules and the work of the Australian AI Safety Institute.

Hugging Face is not an Australian company, but the scenario has been raised locally. Asked at the same event what would have protected an Australian business in the same position, Charlton said Australia had “a lot of agency” and was working with overseas regulators, while declining to discuss specific conversations.

David Swan travelled to San Francisco as a guest of Salesforce.

Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.

You have reached your maximum number of saved items.

Remove items from your saved list to add more.