Why An Anthropic Researcher’s Terminator-Style Warning Caught Fire

An Anthropic safety researcher’s warning Tuesday that his company believes AI “could kill all humans” within a decade has reverberated throughout Silicon Valley, Wall Street and Washington, potentially complicating the Claude maker’s upcoming initial public offering and its efforts to repair ties with U.S. political leaders.

As the comment circulated around the internet and on TV broadcasts, some Anthropic employees and other AI staffers defended their efforts to minimize harm their technology might cause—while acknowledging there was much work to do. Paul Christiano, whom OpenAI just tapped as a board director of a nonprofit that oversees the company, echoed the Anthropic warning in a statement Wednesday, saying that “If we build superintelligence without more robust alignment”—or safety constraints—“I expect we will permanently lose control of it. If that happens then most people could die.”