When tech leaders trade numbers for the odds that advanced AI ends civilization, they are invoking "p(doom)"—a single probability estimate that has become Silicon Valley shorthand for existential risk. But the number itself is the least useful part of the exercise. Researchers at Georgetown's Center for Security and Emerging Technology argue that probability is often the wrong tool for AI risk, and the U.S. National Institute of Standards and Technology offers a process-oriented alternative focused on managing risk rather than guessing at it. Understanding these three approaches—and why experts disagree so sharply—is the practical core of the AI risk debate.
What is p(doom)? Understanding AI's Existential Risk
P(doom) is, as Forbes Australia reports, a Bayesian probability estimate that the development of AGI or superintelligence within a 20-to-100-year window will result in civilisational catastrophe, expressed on a percentile scale from 0 to 100. The term has become commonplace in innovation circles, with numerous tech heavyweights publicly weighing in—though Bill Gates, for one, has not offered his own value, instead suggesting that frontier AI companies may not pull the trigger themselves but have created the ecosystem and tools that could facilitate mass destruction by others.
The definition embeds several contestable assumptions: that "AGI" is a well-defined threshold, that catastrophe arrives within a specified time window, and that a single number can capture the outcome. Notably, when researchers survey experts, they must first pin down what AGI even means—one survey defined it as AI systems better at STEM research than the best human scientists, potentially with other advanced capabilities. That definitional fragility is the first hint that a single probability is doing a lot of quiet work.
The Limitations of a Single Probability: Why Experts Disagree
Why do experts disagree so widely on p(doom)? A survey of AI experts by Severin Field, conducted through a Cambridge ERA:AI fellowship, frames the question directly, noting that leading AI labs and scientists have called for the global prioritization of AI safety, citing existential risks comparable to nuclear war. The disagreement is not merely rhetorical. There is little data or detailed theory to assess AI's potentially catastrophic risks, as CSET's Andrew Lohn observes, so policymakers often resort to expert best guesses.
This creates a compounding problem. When the evidence base is thin, individual estimates reflect prior beliefs and familiarity with AI safety concepts more than measured frequencies. A single number compresses all of that—definitional choices, time horizons, causal assumptions, and personal exposure to the safety literature—into one figure that looks rigorous but obscures its own foundations. Two experts with identical "5%" answers may be describing entirely different risks.
Quantifying Uncertainty: The Belief/Plausibility Model
Lohn's May 2026 CSET report, "Beyond P(doom) for AI Risk: Quantifying Uncertainty Without Probability," proposes an alternative that is mathematically rigorous yet requires policymakers to ask only two simple questions. The approach uses two values: Belief, a lower bound supported by evidence, and Plausibility, an upper bound representing what remains after removing counter-evidence.
CSET illustrates the mechanics with a die. If two of six sides show stars, the evidence supports a 1/6 degree of belief that a roll comes up stars; since two sides cannot be stars, the plausibility is 4/6. The gap between the two numbers is a direct measure of ignorance. Crucially, without ignorance, Belief and Plausibility converge to the same number—ordinary probability. The model's advantage is that it makes uncertainty itself visible rather than averaging it away, which matters most precisely where data is scarce and theory is incomplete, as in AI risk.
Managing Risk: The NIST AI Risk Management Framework
Where p(doom) asks "how likely is catastrophe?" and Belief/Plausibility asks "how much do we actually know?", the NIST AI Risk Management Framework asks "what should we do?" Released on January 26, 2023, the AI RMF is intended for voluntary use to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems. NIST developed it through a consensus-driven, open process that included a Request for Information, several public draft versions, multiple workshops, and other opportunities for input.
The framework is explicitly designed to build on, align with, and support risk management efforts by others rather than replace them. Its orientation is procedural: instead of producing a probability, it structures how organizations identify, measure, and govern risks across an AI system's lifecycle. For catastrophic risks that cannot be confidently quantified, that shift—from estimation to management—is the practical response.
Comparing AI Risk Assessment Frameworks
The three approaches serve different analytical needs, and the table below contrasts them across a common set of attributes so you can match the tool to the question you are actually asking.
| Approach | Quantification Method | How It Handles Uncertainty | Primary Use Case |
|---|---|---|---|
| p(doom) (single-point probability) | Bayesian probability estimate of civilisational catastrophe from AGI, on a 0–100 percentile scale; often an expert best guess given limited data. | Compresses all uncertainties into one number, potentially obscuring underlying assumptions; does not distinguish types or sources of uncertainty. | Communicating an individual's overall risk impression; comparing headline stances among tech leaders. |
| Belief/Plausibility (CSET) | Two values: Belief (evidence-supported lower bound) and Plausibility (upper bound after removing counter-evidence); requires two simple questions. | Explicitly quantifies ignorance as the gap between the two bounds; without ignorance the bounds converge to ordinary probability. | Analyzing risks where data is scarce and theory is incomplete, with transparency about what is known versus unknown. |
| AI RMF (NIST) | No probability output; a voluntary framework for incorporating trustworthiness into design, development, use, and evaluation of AI systems. | Handles uncertainty through structured process—consensus-built governance steps rather than a numeric estimate. | Organizational risk management across an AI system's lifecycle; operational governance rather than prediction. |
The comparison clarifies the division of labor: p(doom) communicates, Belief/Plausibility quantifies what is unknown, and the AI RMF governs. None of them replaces the others, and none delivers a definitive risk value.
Navigating AI Risk: From Speculation to Structured Management
To effectively assess and manage catastrophic AI risks, decision-makers should move beyond single-point probability estimates like p(doom) and adopt more structured frameworks that explicitly address uncertainty and provide actionable governance steps. This is not a prediction about AI outcomes, nor an endorsement of one framework: p(doom) remains a useful conversational shorthand, but the expert disagreement documented in surveys—and the thin evidentiary base CSET describes—means a lone number cannot carry the weight of policy.
The measurable indicator of progress is increased adoption and consistent application of structured AI risk management frameworks (e.g., NIST AI RMF) within organizations developing and deploying advanced AI systems. For readers tracking the debate, that is the concrete signal to watch: whether the organizations building frontier systems can demonstrate structured risk processes, rather than executives quoting divergent percentages whose assumptions no one can inspect.
Sources
- Beyond P(doom) for AI Risk — Center for Security and Emerging Technology
- AI Risk Management Framework — NIST
- What is p(doom)? How tech leaders evaluate existential AI risk — Forbes Australia
- Why do Experts Disagree on Existential Risk and P(doom)? A Survey of AI Experts — ARXIV





