UK reveals AI agreement as Elon Musk warns of extinction

London: The UK government has announced a “world first agreement” on how to manage the riskiest forms of AI.

It focuses on so-called “frontier AI” – what ministers consider highly advanced forms of the tech – with as-yet unknown capabilities.

The agreement, signed by countries including the US, the EU and China, was announced at the UK’s AI Safety Summit.

Ahead of the meeting, Tesla and X owner Elon Musk, who is attending, warned AI could lead to humanity’s extinction.

But other attendees have warned against speculating about unlikely future threats and said the world should instead focus on the potential present day risks AI poses, such as replacing some jobs and entrenching bias.

There has also been a pre-recorded statement from King Charles in which he called the development of advanced AI “no less important than the discovery of electricity”.

“We must similarly address the risks presented by AI with a sense of urgency, unity and collective strength” he said.

The UK government said the Bletchley Declaration, which attendees have signed, has seen 28 countries agree there is an urgent need to understand and collectively manage potential AI risks.

Prime Minister Rishi Sunak said it was a “landmark achievement that sees the world’s greatest AI powers agree on the urgency behind understanding the risks of AI – helping ensure the long-term future of our children and grandchildren.”

Other countries have also stressed the need for a global approach to managing the technology.

Relations between China and the West are fraught in many areas – but the country’s Vice Minister Wu Zhaohui told the conference it was seeking a spirit of openness in AI.

“We call for global collaboration to share knowledge and make AI technologies available to the public,” he told delegates.

Meanwhile, US Secretary of Commerce Gina Raimondo said the US would be launching its own AI Safety Institute following the summit.

UK Technology Secretary Michelle Donelan, who chaired the opening remarks, announced the next summit will be held virtually and hosted by the Republic of Korea in six months.

The next in-person event will then be held by France in one year’s time.

A dire prediction about AI came from Elon Musk, appearing on comedian Joe Rogan’s podcast on Tuesday, where he claimed some people could use AI to protect the planet by ending human life.

“If you start thinking that humans are bad, then the natural conclusion is humans should die out,” he said.

“If AI gets programmed by the extinctionists, its utility function will be the extinction of humanity… they won’t even think it’s bad.”

He was speaking ahead of the UK’s AI safety summit, now under way, where he is due to meet the UK prime minister.

Many experts consider such warnings overblown.

Nick Clegg, the president of global affairs at Meta and former deputy prime minister – who is also attending the summit – said people should not let “speculative, sometimes somewhat futuristic predictions” crowd out more immediate challenges.