Anthropic says AI is developing so fast, the trend points towards systems becoming capable of developing their own successor. We’re not there yet, but it believes it “could come sooner than most institutions are prepared for.” In a blog post, Anthropic explains that AI that can build itself could “bring enormous good for the world” in the fields of science and healthcare, among others. However, it might also “increase the risks of humans losing control over AI systems.” So how can humanity prevent Skynet from becoming a real thing? Anthropic is suggesting that the answer is the global slowdown or the temporary pause of AI development “to enable societal structures and alignment research to keep up with the advance of the technology.”
The company is one of the leading names in the AI race today, and unlike its competitors that have yet to make money, it’s reportedly on track to have its first profitable quarter. It recently filed paperwork with the SEC to go public, likely before the year ends. As The Wall Street Journal notes, critics suggest that Anthropic’s warnings about its own technology are a marketing ploy, perhaps in an effort to make it look like the least egregious among all the AI companies or to make it appear as if its products are the best out there.
Critics are specifically pointing to the limited release of its cybersecurity AI model Mythos as an example. Anthropic said it was making Mythos available to a select group of partners due to the potential damage its ability to quickly identify vulnerabilities could cause in the wrong hands. But people think it’s just a ploy to hype up the product or to cover up the fact that Anthropic only wants to sell it to the biggest enterprises.
It’s worth noting, however, that the company’s suggestion is based on the findings of Anthropic Institute, a research division it established in March. Anthropic said back then that the institute’s role is to “tell the world” what it learns about the challenges that arise as AI firms develop more advanced AI systems. The institute, along with collaborators, will conduct research on what’s needed to “build the systems that a credible slowdown or pause would require.”
If AI companies agree to a slowdown, mechanisms must be in place to verify that they’ve all actually stopped or slowed down AI development. Otherwise, some could jump ahead of the rest after developing their technology in secret. “A meaningful slowdown or pause would require multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions,” Anthropic writes. “It would also require that each can verify that the others have actually stopped.”
Its suggestion would only ever happen if all AI companies around the world can get together and pledge to stop AI development for a while. Anthropic says it’s not impossible, citing nuclear-weapons treaties as an example, though it admits those agreements were decades in the making. We don’t have that long, seeing as AI development is progressing at a rapid pace. The company is planning to hold talks with policymakers, researchers and other AI companies over the coming months over this issue and will publish the results of those conversations.
Stephan is the sports journalist for the Maple Grove Report.
