Anthropic CEO: AI faces a trust crisis because people feel cheated by governments and tech firms

Full text

A lot has already been said about the doom and gloom surrounding AI - some of it drawing on the warnings of classic dystopian novels, but much of it grounded in reality. There is a lot of optimism and promises from those in control of the technology, but not much has emerged in the form of actual benefit.

“I think by far the most accurate criticism of AI companies including Anthropic is that we haven’t yet delivered on our big promises to benefit the world,” Anthropic CEO and co-founder Dario Amodei has said in a post on X .

The technology is suffering from a “crisis of trust” that stems from people’s belief that companies and governments are “cooking up some new way to screw them over,” he argues. But given how much of the US military, law enforcement, and intelligence gathering infrastructure now relies on AI, are the people wrong?

AI can’t public relations its way out of this one

As AI technology took off, much of the benefit of the technology was aimed at convincing executives and C-suites that the technology - while unproven - was a worthwhile investment. As a result, multiple AI leaders positioned the technology as a way to strip away the labor costs of companies by replacing workers with AI.

Some AI gurus are still arguing these points behind closed doors. I attended a private event this year where an AI "guru" predicted that AI superintelligence would erase as much as 50% of the global workforce within 10 years, and the companies best positioned to take advantage of this cataclysmic event would reap unimaginable control and riches. This was naturally met by applause from the upper echelons of management from companies in attendance.

Amodei himself famously said that AI could wipe out 50% of white collar jobs. But he has since restructured this message, saying instead that AI is a ‘multiplier of output’ - or corporate word salad for ‘increasing productivity without having to increase compensation’.

As I have said numerous times, the proof of AI really is in the pudding. We’ve heard everything that it could do, but seen little benefit in what it can do. To his point, Amodei acknowledges that.

He notes that if an advert went up saying that AI could cure cancer, it would be perceived as entirely misleading. “The thing that will work is actually curing cancer ,” he says.

Amodei has teased Anthropic’s morality on these issues, saying that the company is “ramping up its efforts” to focus on using AI for biology and medicine. But again, we’re yet to see the results.

Amodei’s post also touched on where AI is centralizing power, echoing some of Mark Zuckerberg’s recent rant on how best to use AI for everyone’s benefit. Amodei’s argument differed slightly from Zuckerberg’s, arguing that there is a balance to be struck between regulation, oversight, testing, and open-weight models.

He notes that in some aspects he agrees with the “Pacing the Frontier” movement , stating that implementing controls on the frontier labs would “modulate” the pace of the technology while simultaneously allowing the rest of the industry to catch up.

The obvious flaw in this plan is that no other nation would be subject to this regulation which could deliberately hamstring US hegemony in the AI industry - something the Trump administration has frequently said it will not allow to happen.

Is AI delivering more benefits to authoritarian infrastructure than to people?

Where AI systems have been deployed extensively is in surveillance infrastructure. Until the collapse of its relationship with the Pentagon, Anthropic was working with the Department of Defense to procure AI systems. Anthropic refused to cooperate in allowing its technology to be used for mass surveillance and fully autonomous weapons, and lost the contract.

Luckily for the DoD, OpenAI shook hands on a new deal with the Pentagon and immediately filled the shoes left by Anthropic with a prohibition on its technology being used to spy on Americans.

That doesn’t mean other companies are abiding by the same level of ethics. Flock has faced a widespread guerrilla movement targeting its AI-powered cameras designed to track vehicle number plates across the US.

The system is intended to flag suspect’s cars as they travel across boundaries and lines of authority, but has instead seen success as a way for members of law enforcement to stalk exes , monitor colleagues, and track their love interests.

AI systems are also being deployed in traffic monitoring and speeding cameras. This in particular has received a lot of attention as a grey area of policing, especially for systems that misidentify numberplates or are wrongly configured.

For criminal cases in the US, the burden of proof rests with the government to prove someone committed a crime. However, some autonomous ticketing systems that have issued citations to the wrong vehicles have been criticized for requiring those accused to prove their innocence in court, flipping the burden of proof for criminal cases on its head.

And then there is the elephant in the room.

Palantir’s AI technology is deployed across the US military and law enforcement. Most notably, it has been used by the US Immigration and Customs Enforcement (ICE) to track those suspected of illegal immigration - but Palantir’s technology is also used in drone imagery analysis, long-range precision strikes, and military data analysis and planning.

No doubt it has been instrumental in delivering the World Record for the most times a single war has been won.

AI is great news if you have nothing to hide and love being on the side of the Good Guys™, but that's a constantly shifting definition.