KRISTIN WOOD, MARTIN PETERSEN
During her 20-year CIA career, Kristin Wood served in the Director’s area and three Agency directorates – analysis, operations, and digital innovation – leading a wide variety of the Agency’s missions in positions of increasing authority. Among her key Agency assignments were Deputy Chief of the Innovation & Technology Group at the Open Source Center (OSC). She led OSC’s open-source IT and innovation efforts to extract meaning from big data.
Martin Petersen spent 33 years with the CIA, retiring in February 2005 as Deputy Executive Director and Acting Executive Director. In the course of his agency career, he ran two large analytic units (The Office of East Asian Analysis and the Office of Asian Pacific Latin America Analysis) before becoming Associate Deputy Director of Intelligence for Strategic Plans and Programs, the first Chief Human Resources Officer for CIA, and Deputy Executive Director.
OPINION — Last month marked the 20th anniversary of the Iraq War, begun after the September 11th 2001 al-Qaida attacks on New York and Washington DC. The George W. Bush Administration judged that Iraqi President Saddam Hussein’s weapons of mass destruction (WMD) program and his enduring support for terrorist causes posed an unacceptable risk of translating into a WMD-enabled al-Qaida.
The war was justified in part, on Intelligence Community (IC) assessments that Iraq had and was hiding, its WMD and uncertainty about the nature of ties to -al-Qaida. No WMD was discovered, and subsequent Congressional reviews of pre-war judgments revealed that the intelligence reporting was much weaker and the sourcing more questionable than intelligence officers realized or communicated at the time.
The intelligence failure forced the IC to restructure how intelligence is collected and analyzed and how analysis is characterized, work that continues to be refined to this day.
With that as backdrop, we wondered: how would pre-war Iraq analysis be different if we had the analytic tools then, that we have today?
Let’s start with AI. We believe AI can greatly improve intelligence analysis in several ways that are relevant to where and how the IC went wrong. While there are many more developments of relevance, in focusing on just the promise of AI we see the opportunity for:
Saving time and history. Pre-war IC officers worked long hours and struggled to stay atop their data; their successors do the same and are also buried in data that far exceeds a human’s ability to review it. AI tools use machine learning (ML) and natural language processing (NLP) to help analysts process and extract key insights from vast datasets. If AI tools sit atop the data archives, they can also ensure that compartmented or older information continues to factor in to new judgments without being lost or forgotten when analysts or units move or when accesses change, all perennial problems.
What news are you missing today? Get full access to your own national security daily brief by upgrading to Subscriber+Member status.
Pointing the way. AI tools can help identify patterns, relationships, or anomalies that might not be apparent to human analysts, tipping and queuing them towards areas for further investigation. This is useful in any fast-breaking situation, but when caught between the drumbeat of war and vast amounts of data, this capability would have helped the IC tremendously.
Finding bad actors. One of the challenges of the Iraq terrorism analysis was the challenge of combining all of the terrorism-related data to assess the nature of al-Qa’ida’s presence in Iraq. An AI tool could have helped us make linkages among those we knew to find others we did not yet know.
Opening our brain’s black box. For us, one of the most exciting areas of AI’s potential is its ability to make an analyst’s or group’s internal decision-making processes more transparent.
Tools can help by showing how we weigh key elements of argumentation and evaluate the importance of such variables, allowing us to better see missing elements of rigor in analytic thinking and identify gaps that are limiting analytic confidence. It can also allow analysts and managers to track how thinking has evolved over time and highlight critical differences, avoiding both group think and “layering,” a term describing multiple points of analysis added to others without the right caveats, both of which were failings in pre-war WMD analysis. Having data-driven insights inform analytic frameworks also would allow for more reasoned and less emotional discussion of the evidence.
Helping us understand what we see. One of the more compelling points in then-Secretary of State Colin Powell’s speech to the UN explaining the Administration’s reasoning came through an image of an alleged mobile biological lab facility; it was later determined that the vehicle had been wrongly identified. Today’s computer vision machine learning-assisted models could have offered better analysis through millions of ground photos and overhead images and tracking vehicle locations to develop a more accurate assessment of both the equipment and its transit history.
For all of its promise, AI tools cannot yet do deliberative thinking, referring to planning and evaluating. AI also can’t communicate, a key human ability. Those shortcomings are obvious in ChatGPT which attempts to automate reasoning but doesn’t “think” critically, instead regurgitating plausible-sounding responses based on large language models.
The Cipher Brief hosts expert-level briefings on national security issues for Subscriber+Members that help provide context around today’s national security issues and what they mean for business. Upgrade your status to Subscriber+ today.
Today’s AI can answer the questions we ask of it, but it cannot ensure we’re asking the right questions. This was a failing of the Iraq WMD NIE, answered the question: what is the evidence that Iraq has WMD?” If we had instead asked “how confident are we that Iraq has WMD?” or “what do we know about Iraq and WMD?” the NIE might have offered much more nuanced judgments.
This matters. Twenty years ago, an intelligence failure was a key factor justifying sending US troops to Iraq, kicking off a series of events that resulted in the execution of the Iraqi leader, a restructuring of the Middle East, the rise of the Islamic State of Iraq and the Levant (ISIS), the loss of tens of thousands of lives, and came in at costs estimated to be greater than a trillion dollars.
We believe that the most important element in modern intelligence analysis remains the people who ask the right questions, master their tradecraft, tools, and data, and use them to deliver decision-advantage to US national security officials. The IC’s serious and ongoing focus on advancing analytic skills combined with significantly expanding access to AI, other analytic tools, and vast troves of meaningful data will arm them with the right weapons to protect our national security in the digital age.
No comments:
Post a Comment