# Where’s the Evidence for Doom?

No AI system has shown an independent intention to wipe out the human race. Researchers have watched models lie, game metrics, and try to avoid being shut down in lab tests. That stuff is real. But there's a world of difference between that and an autonomous drive to eliminate humanity.

That is the actual record, and it should be the starting point of the discussion.

Instead, a growing number of people, including people inside the companies building the systems (Doomers), talk as if human extinction is a live, near-term, high-probability outcome of continuing to build AI. Some quit over it. Others say we are approaching a point of no return. The media treats the scenario as a serious forecast instead of an untested extrapolation, and people act like it's already settled.

Something's gone wrong with the burden of proof. People are making an extraordinary claim: that further AI progress will produce machines that seize power, resist control, and potentially exterminate the species. Yet the people who question that claim are increasingly expected to prove it won't happen. That's backwards. The burden sits with the people making the prediction, not those questioning it.

The argument is familiar - once systems get capable enough and get broad autonomy to pursue long-term goals, certain intermediate objectives become useful no matter what the terminal goal is. Staying on, getting resources, gaining influence, stopping humans from interfering. At high enough capability, humans become obstacles. The system doesn't need to hate us - it only needs to be better at its objective than we are at stopping it.

That's a coherent theoretical story, not an observed sequence of events. Huge difference.

The story has a history: I. J. Good on the intelligence explosion, Vinge on the singularity, Bostrom's Superintelligence, Yudkowsky and others. For a long time it stayed inside small communities. Then AI capabilities jumped and the abstract scenario suddenly felt a lot more real. The models got much better, fast. Nobody proved that better models would lead to persistent power-seeking, irreversible loss of control, and extinction - those are separate claims, each requiring its own evidence. Instead, Doomers started treating raw capability progress as evidence for the whole chain.

The problem is that the extinction argument is built so almost nothing can count against it. Today's systems don't try to kill us, they cooperate with supervision, and they stay controllable, and the answer is always that they just aren't powerful enough yet. Because the theory puts the dangerous threshold somewhere in the future, the total absence of extinction-seeking behavior fits right in.

That doesn't automatically make the theory false. Plenty of real predictions are about things we haven't seen yet. Still, if a theory can explain away a lack of evidence, that lack of evidence doesn't magically count in its favor. When a framework can absorb almost any outcome, you need more than a good story before you start treating it as likely.

Notice how one-sided the scoring gets. When an AI acts deceptive in a contrived lab test, people point to it as evidence that dangerous autonomy is coming. When millions of ordinary runs produce no attempt to seize power or harm anyone, those get brushed off because the dangerous capability supposedly hasn't arrived yet. Models improve, and that gets counted as proof we're getting closer to the edge. Catastrophe doesn't happen, and the edge just gets pushed further out. Explaining things after the fact isn't the same as predicting them, and pointing out that something could go wrong doesn't mean it probably will.

The size of the downside doesn't fix this. Once you frame the outcome as the end of the species, even a tiny chance can feel like a reason to panic. Sure, the stakes are huge, but how bad a disaster would be tells you nothing about how likely it is. A one-in-a-million shot and a fifty-fifty shot aren't interchangeable just because both end terribly. If someone claims the odds are high, they need a real basis for that number. Pointing at how awful the disaster would be doesn't supply it.

That exact math also helped Effective Altruism (EA) build real institutional power around a speculative worldview. EA started out trying to make charity more rational, but its longtermist wing decided protecting humanity's distant future mattered more than almost anything else, and AI extinction became their main cause. We're not talking about random cranks on the internet. EA founder Toby Ord put the odds of existential catastrophe from unaligned AI this century at 10%. Co-founder William MacAskill estimated around a 3% chance of misaligned AI taking control of civilization. Eliezer Yudkowsky and the overlapping rationalist crowd were even darker. Not everyone in EA is a Doomer, obviously, but some of its biggest names spent years warning that AI could end humanity.

More to the point, they put serious money and organization behind it. Long before the rest of tech cared about alignment, EA donors were bankrolling it. Open Philanthropy, backed by Dustin Moskovitz and Cari Tuna, gave OpenAI $30 million back in 2017 and put its co-founder Holden Karnofsky on the board. It funded independent safety researchers, university labs, and policy shops. Operating as Coefficient Giving, it committed $351 million in 2025 across technical AI safety, security, and field-building, and in September 2026 reported being on track to commit over $1 billion for the year. At the same time, EA groups built the talent pipeline. 80,000 Hours has been steering people into AI risk careers since 2014 and made it their #1 global priority starting in 2016, while Open Philanthropy funded fellowships placing people into congressional offices, government agencies, and think tanks.

So when ChatGPT dropped and everyone started freaking out about AI, guess who was already standing there with credentials, talking points, and a ready-made hiring pool? Suddenly labs needed safety teams, and lawmakers and journalists needed experts to call. The EA and rationalist networks had spent a decade preparing for that moment, giving them a massive head start in shaping the debate. None of this is some secret conspiracy. It happened in broad daylight through grants, hiring, and advocacy. And I'm not saying these people are faking it. Plenty of them genuinely believe they're saving the world. Being sincere just doesn't prove you're right.

Open Philanthropy was upfront about how its funding philosophy worked. Back in 2017, Karnofsky explicitly argued that their brand of high-stakes giving called for "less of a requirement for evidence" than traditional charity. For philanthropy, that's a defensible call. If you want to place long-shot bets on huge hypotheticals, go for it. Just don't confuse deciding a risk is worth funding with proving the risk is actually coming.

The trouble starts when that funding model kicks off a self-reinforcing loop. Money pays for institutes and fellowships, producing a whole class of AI safety experts who write papers, advise politicians, and get quoted in the press. Pretty soon, outsiders see a massive professional ecosystem and a tight-knit community agreeing with itself, and they mistake it for independent validation of the assumptions that created those jobs in the first place. Thousands of smart, well-funded people can build entire careers around a hypothetical disaster that never happens. A billion dollars in grants and a pile of credentials still don't turn a guess into a fact.

Social pressure reinforces all of this. In some circles, acting terrified about AI makes you look serious, while asking for proof gets you labeled naive. The media obviously loves the scariest version - "AI might create hard control problems" gets way less attention than "AI could kill everyone." So a speculative argument gets repeated louder and louder until the urgency itself starts passing for credibility, and a hypothetical about future machines starts sounding like an established fact about where the technology is going.

The underlying control questions are real, though. Very capable systems could genuinely be hard to manage. Models have already found loopholes in their rules, behaved deceptively, and done unexpected things in lab tests. As they get more autonomous and take on bigger jobs, those problems need real engineering work. Being cautious under uncertainty makes sense, and requiring labs to test their systems and show they're safe before critical deployment is completely reasonable.

That practical case for caution isn't the same as proof of an impending catastrophe. Asking builders to show a system is safe before shipping it is a normal engineering standard. Declaring that human extinction is probably coming and demanding skeptics prove otherwise is a totally different move. Saying "this could be dangerous and we should figure out why" is just common sense. Saying "this is probably going to kill us all" is a bold prediction, and it needs real evidence.

Nobody can prove that no future AI will ever become dangerous or develop goals opposed to human interests. There are too many ways the technology could go to rule anything out for good. That doesn't mean we have to treat every worst-case scenario anyone can dream up as an approaching reality.

The basic fact hasn't changed: no AI system has ever been shown to independently pursue the extermination of humanity. The theoretical arguments for why a future system might are worth examining, but they're still just theory, and nobody actually knows the odds of the worst case. If someone thinks human extinction is the likely outcome of AI development, they can lay out the mechanism, show what backs up their numbers, and tell us what evidence would actually change their mind.

The burden of proof belongs to the Doomers. They're predicting the most extraordinary catastrophe imaginable, yet somehow the people asking for evidence are expected to prove it won't happen. That's backwards. Until they can demonstrate why extinction is likely, they're entitled to their fears, not to have those fears treated as facts.
