Anthropic's prospectus warns of AI's 'existential risks to humanity'

The prospectus, circulated with a select group of partners, used up almost one-third of its S-1 filing to detail the 'risk factors', among which it mentioned the potential of advanced AI models to 'manipulate, blackmail and exhibit other unpredictable behaviour', the report stated.

 

Anthropic has formally warned investors that its technology may pose "existential risks to humanity" in a long-awaited initial public offering prospectus circulated with a small group of partners in recent days. 

The development comes ahead of the leading US group launching what the markets expect to be the most highly valued IPO of all time, the FT report stated.

However, these warnings in its prospectus add to the challenges in valuing Anthropic by public market investors.

Claude's maker also provided investors with a clearer picture of the challenging economics of building state of the art AI models ahead of the listing, expected on the Nasdaq this autumn.

Anthropic's backers are hopeful it can list at a valuation of more than $2tn, twice what it achieved in May. Claude's maker plans to spend $518bn on cloud, computing and infra to support fast-paced growth in the coming years, the report stated.

The dark lining to the silver cloud, however, comes from public warnings issued by ex-employees that unbridled AI could end humanity in a decade.

Amodei, who addressed the UN Security Council last week, warned the world that AI could threaten humankind and was "the most important global security issue facing the world today".

Sam Altman, his arch-rival at OpenAI, has also faced the heat with mounting safety concerns after AI "agents" have breached external systems.

Only last week, OpenAI disclosed that its tools had hacked "dozens" of external sites, including governments, and cited safety concerns for scrapping its plans to release its newest model next month.