Last week I attended AGI.WTF, “an event for making sense of what is happening with AI.” The conference did more or less what it set out to do: describe what’s been happening in AI the past few months from the perspective of people in the AI safety community.
I got back on Thursday. On Sunday, the WSJ published an article on about the folks in AI safety scene titled “The Doomers Who Shaped the AI Safety Freakout.” They mentioned the conference venue by name! There are suddenly a lot of people talking about Effective Altruism, Bay Area “rationalists,” and the credibility or lack of credibility of those folks.
As someone on the periphery of this scene who’s neither an Effective Altruist or a rationalist, I want to try to bridge the gap between what I see reported and what I’ve seen myself. I notice a few things about these folks that I don’t see reflected in news coverage.
For one, the doomers aren’t even doomers. A weird thing about the term “doomer” is that it’s used to describe anyone who thinks AI could be dangerous. Only some of those people predict doom. Nate Soares spoke at the conference; his book is called “If Anyone Builds It, Everyone Dies.” Fair enough to call him a doomer! Then there are people who think the risk from AI is pretty low — say, below 10%. And there are people who think the risks exist but can be mitigated with good regulation, but notice that we don’t have good regulation yet. “Doomers” is not a useful word to describe this entire group!
I found that speakers and attendees at the conference share a pretty simple set of propositions:
AI is getting smarter and smarter, and better and better at doing stuff.
AI reaching superhuman levels of intelligence could do a lot of harm.
I wasn’t surprised to hear disagreement among folks I talked to about how likely it was that AI would go rogue. What was surprising was that the disagreement didn’t seem very important. If ASI is possible, the argument goes, we should support slowing down AI development so it’s done safely.
I’ve noticed that my friends outside the Bay Area who oppose AI tend not to believe AI will become capable enough to actually be dangerous. I understand why folks are suspicious of the hype around AI. And I can also understand being suspicious that a bunch of weirdos in the Bay Area think AI will be not just capable, but also dangerous.
Nate Soares was on a panel with Ed Zitron a couple weeks ago. Zitron has been a prominent skeptical of AI’s capabilities and of the finances of the AI labs. I know Zitron thinks the AI safety stuff is a distraction; Soares thinks AI has a high likelihood of going rogue. I asked Soares about the panel and about Zitron in particular, and Nate said they’d had a good conversation. Soares wants to see the AI companies get totally shut down; I think Zitron does, too. My hope is that people like Soares and Zitron can become allies, even if they have some substantive disagreements. They don’t need to agree on AI risk, or even about AI capabilities, to work together.
Among folks at the conference, there was a lot of effort focused on getting lawmakers to make laws to restrict AI development, both within California and nationally. There are also people trying to think about how to make AI development / deployment happen in ways that are better for society than, say, social media and cell phones. Some part of that is showing people how much AI capabilities are increasing, so we can make laws that run ahead of the technology (or slowdown/ban the technology) rather than responding to technology after the fact.
I am convinced that AI capabilities are increasing, and will keep increasing at a very fast rate, and that seems like an important thing to understand so we can collectively figure out what to do. If we underestimate how capable AI is and will be, we will make bad predictions about AI companies and make public policy that is inadequate in unpredictable ways.