26 October 2022

How not to estimate the likelihood of nuclear war

Amy J. Nelson and Alexander H. Montgomery

As Russia retaliated for Ukraine’s destruction of the Kerch Bridge by launching strikes on energy facilities and civilian targets in Kyiv, commentators returned to the question of whether events were escalating, and whether the world was inching closer to the brink of nuclear war. Probability estimates by these observers have, unsurprisingly, mushroomed as well.

On the high end, these estimates ranged from 10-20 percent to an overly precise 16.8 percent to 20-25 percent for “some analysts.” Some of these headline-grabbing estimates are likely inflated to create a sense of urgency and put pressure on policymakers to take action, rather than to showcase the ability to carefully craft probability estimates. The difference between estimates may simply reflect the prominence of each nuclear scenario in each analyst’s mind.

Here, we lay out the debate over the probability of nuclear use, outlining flaws in current estimates. We offer an alternative approach that focuses on thinking broadly across multiple scenarios and minimizing the rewards of using nuclear weapons, to minimize the possibility of nuclear war.

CAN YOU PUT A NUMBER ON THE LIKELIHOOD OF NUCLEAR USE?

Predicting the future is hard, and estimating the probability of future events is no exception. Estimating the probability of future events like ones that have occurred many times before is already difficult enough, since — in a complex world — it is difficult to determine which factors decreased or increased the probabilities for past events. For example, the possible causes of Russia’s attacks on Ukrainian civilians span across international, domestic, and psychological explanations.

Trying to estimate the probability of nuclear use is just as difficult — if not more so. Nuclear scholar and Russian forces expert Pavel Podvig argued along these lines in Newsweek and on Twitter, publicly asserting that nuclear strikes are so rare, it is impossible to calculate their frequency and therefore meaningless to translate that frequency into a probability. Director of the Global Catastrophic Risk Institute Seth Baum noted that Podvig was critiquing frequentist approaches to calculating, which infer how likely an event is to happen based on a sampling of past similar events. He argued that instead, Bayesian approaches – which rely on subjective probabilities that get updated when new information is presented – could be a more helpful way of thinking across multiple different nuclear scenarios, as Baum himself has done in a working paper. Superforecasting, when ordinary people cultivate their intuitive sense for prediction, and which relies in part on good Bayesian updating, is one such approach to nuclear scenarios; Baum’s co-author and superforecaster Robert de Neufville argued in March that there was a 4% chance of at least one fatality from nuclear use by July 1, 2022.

What’s going on here? Is Baum correct that Bayesian approach is the right way to think about nuclear use? And is this war of the nerds helpful in understanding how to avoid nuclear war?

Podvig and Baum are correct that frequentist approaches are definitely not useful here. Estimating the likelihood of a future nuclear war based on how often past nuclear wars occurred is not appropriate, given how rare past nuclear weapons use is.

Bayesian approaches are useful for thinking about adjusting one’s own subjective estimate of probability, but not for informing the actual probability of Putin deciding to use a nuclear weapon. What’s more, there is no way of adjudicating between subjective estimates, and consequently no way of coming up with a combined estimate overall. Even people with the same information may have wildly different guesses: Former U.S. President John Kennedy estimated the odds of nuclear war during the Cuban Missile Crisis to be between one in three and one half, but former U.S. National Security Advisor McGeorge Bundy thought it was one in 100.

Even approaches that attempt to deal with this problem of competing subjective estimates, such as a team of superforecasters working together, cannot work here, since they rely heavily on good estimates of the base rate of an event occurring, which is similarly difficult. As RAND economist Alain Enthoven said to a senior Air Force general, “General, I’ve fought just as many nuclear wars as you have.” Moreover, there is no one in the Kremlin turning the dial up or down, creating an objective change in the probability of nuclear weapons use.

We are also likely to overestimate the likelihood of nuclear war when those estimates are informed by public behavior, because we can’t see the private behavior that would decrease our estimates. Russian leaders have good reasons to make multiple public threats. By increasing other countries’ perception that Russia is willing to use nuclear weapons, the Kremlin increases Russia’s bargaining leverage (although this can also create a commitment trap, where a country that has threatened to use nuclear weapons could be forced to do so in order to remain credible). This means that when numerous threats are made over time, subjective estimations of the probability of nuclear war act as a ratchet, even when the exact same statements are repeated. These estimations cannot take into account unobserved private efforts that are being made to decrease the chance of war.

While we tend to think of nuclear escalation as a ladder, it can be more like an escalator or a vortex, or even a roller coaster. What looks like a terrifying downhill rush towards nuclear war may be balanced out by private counters that slow momentum and return it to level ground: The anticipation of the danger of nuclear use leads actors to try to overcome it. Such recalibrations may be better carried out privately. As U.S. Secretary of State Jake Sullivan noted, “We have communicated to the Russians what the consequences would be, but we’ve been careful in how we talk about this publicly, because from our perspective we want to lay down the principle that there would be catastrophic consequences, but not engage in a game of rhetorical tit for tat.”

It is a mistake for analysts to ascribe a statistical probability for the likelihood of nuclear war, instead of a relative claim (such as “very” or “more than yesterday”). They should also be accompanying estimates with their confidence in the estimate. Yet, many foreign policy experts appear to be far too confident in their assessments. Making a useful probability assessment is impossible under conditions of uncertainty, when we are lacking information about the variety of possible outcomes or the probability of those outcomes.

INSTEAD OF FOCUSING ON NUMBERS, THINK ABOUT POSSIBILITIES

A possibilistic approach, which takes for granted that something might happen but with a likelihood that cannot be determined, is more appropriate here, just as it is when thinking about other high-impact future events. The origins of thinking about the possibility of nuclear war in this way are found in early game theory approaches (economic studies of interacting choices where the outcome of a country’s action depends upon other people’s actions), which offered frameworks that allowed us to think clearly about conditions for nuclear war. But these kinds of approaches can also mislead us if we think we can actually estimate probabilities, or that the estimates we produce must be universally accepted.

Rather than provide punditry on the probability of Putin’s use of nuclear weapons, we should consider pathways that lead to war and reduce the possibility of going down those pathways. In arms control, this means eliminating particularly destabilizing weapons systems; here, it means minimizing the reward of using nuclear weapons. Apocalyptic scenarios have no reward. Yet, limited-use scenarios that help to achieve Putin’s goals may appear to him to do so, even though such strategies are high variance and are essentially “gambling for resurrection” when an actor feels like they are losing and are calculating from the “domain of losses.”

Public and private actions that decrease any apparent value of limited use of nuclear weapons for Putin should, consequently, be the main focus. Since nuclear weapons offer no great advantage for tactical use due to the dispersal of forces on the modern battlefield, use is more likely to be to demonstrate resolve and preserve his strength at home and abroad to counter existential threats to his regime. Consequently, there must be assurances that NATO does not intend to threaten his regime now, and intimations that it may do so if nuclear weapons are used. Western leaders have been careful to state that they do not seek regime change, and should continue to do so. Carefully crafted statements that their war aims and means may change if nuclear weapons are used will help to minimize any benefits Putin might perceive from nuclear use. For example, NATO Secretary General Jens Stoltenberg stated that “Any use of nuclear weapons would fundamentally change the nature of the conflict, and have severe consequences.”

So, if an expert offers you a probabilistic assessment of the likelihood of nuclear war breaking out, you should be very skeptical. Smarter questions and answers should instead focus on scenario-driven approaches that offer different pathways useful for reducing or eliminating certain scenarios. Even though we do not think that quantifying the probability of nuclear use is useful for informing U.S. policy, thinking about how to reduce the possibility of the most salient potential pathways to nuclear use is the best approach.

No comments: