China and Iran used Anthropic’s AI to aid spying, the company says

The report has raised serious alarms

Anthropic, in the report, said that it shut down multiple cases of state-sponsored surveillance operations that used its AI models.

The AI company also warned that governments and state-aligned actors are increasingly using AI to spy on ethnic minorities and dissidents.

What the report by Anthropic revealed

According to the report, the incidents were found and disrupted between January and July, and they originated in China, Iran and west Africa.

Targeted in these surveillance campaigns, as per Anthropic, were “the same diaspora and dissident communities these regimes have historically targeted”.

“These include pro-democracy figures in Hong Kong, Tibetan and Falun Gong communities across Asia, and Iranian minority communities and opponents of the Iranian regime abroad,” the report said.

Iranian actors developed a way to identify people through their social media accounts in one of the cases, while in another, a contractor working for Malian national security authorities used Claude “to design the underlying software that enabled the intelligence gathering”.

Claude CEO Dario Amodei in July called on the US government to support a coordinated slowdown in the development of advanced AI.

The report said: “AI is now being used in place of an engineering workforce.”

Efforts by users to design weapons and create dating scams were also disrupted by Anthropic, which also accused Chinese developers of deceptively using Claude to produce responses to user queries while simultaneously “distilling” that data to improve their own models.

In the industry, “distilling” is a term that refers to a process where one AI model is used to train another one.

Previously, developers from China were accused of using that technique without permission.

Essentially, they stole resources from American labs such as OpenAI and Anthropic.

US companies are “gambling with our lives”, an AI researcher said

An artificial intelligence researcher who left OpenAI to join Anthropic has decided to leave the industry earlier this week.

He accused both US companies of “gambling with our lives” in the race to develop AI models capable of self-improvement.

On X (formerly Twitter), Anthropic's Jacob Coxon wrote that “No other human activity poses this level of danger”.

Coxon spent three years carrying out pre-training research at Anthropic and OpenAI, and he announced his departure from the industry in a lengthy thread on X.

He claimed that neither company is behaving responsibly as AI capabilities continue to advance.

“They are racing straight to self-improving superintelligence and gambling with our lives”, he warned.

How AI might end humanity after global threat was issued

On Tuesday, AI researchers revealed that the chances of the technology killing off humanity in the next decade aren’t exactly zero, an acknowledgment that shocked millions around the world.

While fears of an AI doomsday have existed for years, these recent warning are hitting close to home considering AI has become more powerful and embedded in our daily lives.

According to the experts, there are two broad paths to catastrophe: humans weaponizing extremely powerful AI, or humans losing control of it.

AI could, in these scenarios, make it easier for bad actors to create biological, chemical or other weapons.

On the other hand, rogue systems could attack infrastructure at unprecedented scale while evading human oversight.

A powerful rogue AI could, in theory, evade oversight, replicate itself and resist attempts to shut it down.

In theory it could also develop biological or chemical weapons for nefarious actors, commit a massive cyber attack that cripples humanity's infrastructure, or centralize power in a way that would bring down civilization.

Recursive self-improvement is another dreaded pathway, through which an AI helps build supercharged versions of itself before humans can understand it.

Humanity’s chances of containing or monitoring it before it acts out against humans could be limited if this theoretical AI was misaligned from human goals.

Speaking to Axios, OpenAI CEO Sam Altman said that models are developing quicker than humans can anticipate.

“These models are getting superhuman in many of their capabilities, and we are just sailing in unknown waters”, he said.

But in spite of these warnings from experts, no scientific way can determine the odds of AI killing off humanity.