Are the AI industry’s apocalyptic warnings just a marketing ploy?
Anthropic CEO Dario Amodei told the UN that if “managed poorly,” AI models “could be a risk to humanity as a whole.” | John Lamparski/Bloomberg via Getty Images
Silicon Valley has long warned that its favorite technology might get us all killed. But chatter about AI’s apocalyptic potential has grown louder in recent weeks.
In September, Anthropic researcher Jacob Coxon resigned from the company, accusing his former employer of “gambling with our lives” by developing evermore powerful models faster than it could control them. One of Coxon’s former colleagues vouched for this assessment, saying that there was a greater than 10 percent chance of AI “killing all humans” within the next decade.
Recent pronouncements from the leaders of these companies have been nearly as grim. At the United Nations last month, Anthropic CEO Dario Amodei warned that, if “managed poorly,” AI models “could be a risk to humanity as a whole.” OpenAI’s Sam Altman similarly declared that our species just might “lose control of the future” to the type of software that his firm is selling.
All this triggered a tsunami of media coverage and policy debate about the “existential risks” of AI — and what, if anything, we can do about them.
Key takeaways
Some critics argue that AI leaders’ warnings about existential risk are a PR strategy to attract investment, distract from present-day harms, and shape regulation in their favor.
But this theory doesn’t make a ton of sense.
The simpler explanation is that many AI executives, researchers, and defectors genuinely believe the technology poses catastrophic risks, based on real trends in AI’s growing power and autonomy.
Yet some of the AI industry’s biggest critics are more annoyed than afraid. As they see it, the world isn’t trembling on the precipice of robo-annihilation, so much as falling for the tricks of snake oil salesmen. This argument has been percolating for years, voiced by a broad set of observers, including the linguist Emily Bender (“It makes the product seem more powerful”), computer scientist Timnit Gebru (“It’s not just a distraction, but it’s harmful”), as well as Substack columnist Ed Zitron and various reporters and social media influencers.
In their thinking, Altman and Amodei aren’t talking up their technology’s catastrophic potential because they’re rationally afraid of where AI is going — they’re doing so because their firms are ravenously hungry for more investment. And the reason why “AI doom” has grown more salient in recent weeks isn’t that the case for concern has gotten stronger, but rather that OpenAI and Anthropic’s IPOs have gotten closer.
In short, these Big Tech skeptics argue, journalists and politicians have fallen prey to a counterintuitive PR strategy, one that stokes media frenzies over ludicrous sci-fi scenarios in order to hype AI’s technological promise, distract attention from chatbots’ present-day harms, and help labs secure favorable regulatory treatment.
This would be reassuring, if true. Unfortunately, the theory is unpersuasive. There is little reason to believe that the panic over AI’s “existential risks” was cooked up in a Silicon Valley marketing department. To the contrary, the most plausible explanation for the AI sector’s doomsaying is also the most straightforward: Many of the industry’s executives, employees, and analysts genuinely believe that AI progress poses catastrophic risks.
In my conversations with close observers of the industry, even some of OpenAI and Anthropic’s fiercest detractors conceded that their leaders’ avowed concerns are unlikely to be mere marketing tactics. “The more I learn about the underlying ideologies out of which those two labs were formed, the more I think what we have here is a sincere belief,” the author and computer scientist Cal Newport told me.
This does not mean that Amodei and Altman are right, much less that we can trust in their judgment or integrity. There are many reasons to doubt that the weather forecast for 2036 is “cloudy with a chance of robot apocalypse,” or that anyone can assign a “10 percent” probability to the latter outcome without pulling numbers out of thin air.
Nevertheless, Silicon Valley’s Cassandras have some logic and evidence on their side. AI models are growing both more capable, autonomous, and arguably unruly. If these trends continue unchecked, it’s not hard to see how they could yield very bad outcomes.
Given the stakes, we should not be comfortable dismissing their warnings on dubious grounds. And the “PR” theory of AI doom does precisely that.
How AI companies (supposedly) profit off fears of the robot apocalypse
Before examining why AI “doomerism” probably isn’t a PR stunt, it’s worth saying a bit more about why some believe that is.
According to Bender, Gebru, and others, AI companies benefit from talk of artificial intelligence’s existential risks in at least three ways.
First, it attracts investment. In the words of The Atlantic’s Matteo Wong, the doomer narrative implies that AI labs’ models are “destined to become terrifyingly capable,” and thus massively profitable. As the Los Angeles Times’s Brian Merchant put the point during the first big wave of AI panic in 2023, “what better way to generate a buzz than to insist, with a certain presumed credibility, that your new technology is so potent it might unravel the world as we know it?”
“If you position these threats as all-powerful, as somewhat unknowable, you get to be the expert on it.”
Kate Brennan, AI Now Institute
Second, hyping AI’s hypothetical future hazards diverts attention from the companies’ current malfeasance. Every minute we spend debating whether AI will one day turn us all into paper clips is a minute we aren’t discussing the AI labs’ alleged copyright violations, pollution, labor violations, and myriad other social harms.
And third, the doom narrative shifts regulatory scrutiny away from topics that directly threaten the AI companies’ profitability — such as copyright enforcement — while also suggesting that regulators must rely on the labs themselves for guidance.
“If you position these threats as all-powerful, as somewhat unknowable, you get to be the expert on it,” Kate Brennan of the AI Now Institute, a progressive think tank focused on artificial intelligence’s near-term harms, told me.
Critically, for the biggest skeptics of “AI doom,” industry executives and employees aren’t the only ones complicit in this PR strategy. Self-styled whistleblowers like Coxon are also suspect, since they might be “using the spotlight to attract funding to start a new venture or to secure new positions.” And even nonprofit AI safety organizations are thought to function as vessels for Big Tech’s marketing campaigns, a fact reflected by the financial and interpersonal ties between the major labs and such think tanks.
There are better ways to raise money than promising to end the world
The “PR” critique of AI doomerism has some kernels of truth. AI companies probably would prefer to talk about their technology’s hypothetical future dangers than about their products’ present-day harms. And if publicly touting AI’s existential risks were antithetical to Anthropic and OpenAI’s business goals, their leaders would almost certainly say less in public about that subject. (Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent.)
But it does not follow that Altman and Amodei’s warnings are a self-conscious marketing strategy, born of cynical calculation rather than genuine concern. And there are several reasons to doubt that charge.
For one thing, AI companies have no shortage of ways to hype up their technology’s capabilities. And it is hard to see how any firm could conclude that the optimal approach — from a purely financial standpoint — would be to portray itself as a threat to human survival.
For investors, the appeal of AI companies is not that they could hypothetically get us all enslaved by a machine God; it’s that they could theoretically make a lot of money.
Perhaps casting your chatbots as an “existential” risk will persuade some capitalists of your future profitability. But this seems like an awfully indirect pitch — and one that’s almost certain to be less effective than demonstrating that a lot of corporations would like to rent your software.
“If these guys were selling flypaper — and the revenue was growing as fast as it is — investors would be more or less just as interested,” Roy Bahat, the head of Bloomberg’s venture capital arm, told me.
And even if these firms were hellbent on awing the public with grandiose predictions of their technology’s future powers, they’d have little need to invoke the robot apocalypse. Touting AI’s potential to cure cancer, abolish poverty, or facilitate intergalactic travel would all seem to advance the same objective, while doing less to jeopardize the companies’ public reputations or freedom from intrusive regulation.
And in fact, many with a vested interest in the major AI labs’ success talk up the technology in precisely this way. OpenAI investor Marc Andreessen has decried AI safety concerns as a “full-blown moral panic” while insisting that artificial intelligence will empower humanity to cure all diseases and engage in “interstellar travel.” Similarly, Jensen Huang, the CEO of chipmaker Nvidia and major investor in both OpenAI and Anthropic, has called claims of AI’s catastrophic potential “complete nonsense.” If hyping AI doom is essential for stoking investment, not everyone in Silicon Valley has gotten the memo.
The media doesn’t seem that distracted
Promoting fears of AI’s existential risks would be a similarly odd approach to media management.
Without question, corporations always seek to direct news coverage away from their greatest legal and ethical liabilities. But companies can try to secure more favorable press in many different ways. And implicating themselve
