Last month in The Strategist, Mark Gilchrist put down a wager that computers will ‘be unable to provide any greater certainty than a team of well-trained and experienced analysts who understand the true difficulty of creating order from chaos’. While I commend Mark’s bravery in predicting the future with such certainty, I suspect that, in time, he’ll lose his money. I’d also argue that his zero-sum perspective sets an impossible standard for human analysts and algorithms—whether basic or self-learning. Reducing the intelligence problem down to ‘making sense of war’s inherent unpredictability’ doesn’t do this field of endeavour any justice.
Discussing ‘intelligence’ theory and practice is made all the more difficult by the absence of any universally accepted definition. Nevertheless, talking about intelligence processes and outputs without referring to any intelligence theory leads to inherently inaccurate assumptions—a point Rod Lyon and I made last year in separate Strategist posts.
I’m firmly in Mark’s camp when it comes to the importance of qualitative analysis and the analytical ability of intelligence professionals to make assessments with incomplete datasets. But to do that work, intelligence analysts must have a clear understanding of the epistemological construction for their analysis: they must know what it means to know. Good intelligence tradecraft involves employing a range of analytical techniques to ensure that the validity and reliability of different assessments and explanations are tested.
Mark’s argument against the value of big data analytics and artificial intelligence doesn’t engage with the reality that the role of intelligence is to reduce uncertainty. Seldom is intelligence—be it secret (in the sense that it’s not publicly available) or otherwise—able to offer a decision-maker complete certainty. Rather, an intelligence assessment should be considered as an evidence-based hypothesis accompanied by an associated estimate of its probability: something is probable, likely, possible or unlikely, for example.
In practice, though, the intelligence problem can be categorised as either a puzzle or a mystery.
An intelligence puzzle is a problem you can solve if you’re able to collect and collate enough information. For example, with enough information, you could locate a certain Australian citizen fighting in Syria. The challenge for the intelligence manager is that analysts now have access to an unprecedented quantity and variety of raw data. Sifting through that deluge of data in the required timeframes is now, more often than not, beyond the capacity of a single intelligence professional. With an increasing number of analysts collating data, the task of joining the dots between disparate data points is ever more difficult. Unsurprisingly, increasing the number of data collators may not result in any tangible improvement in output or outcome. Fortunately, that is the exact problem for which AI and big data analytics are best suited.
In contrast to puzzles, intelligence mysteries can’t be solved by gathering more information. Getting to the bottom of a mystery requires analysis and judgement. As highlighted by Rod Lyon, that is the realm of the subject-matter expert. It’s there that intelligence professionals earn their money, and big data analytics may help by supporting hypothesis development and testing.
Mark is again onto something when he argues that the qualitative intelligence analyst is an important component of intelligence capability—and will remain so. However, in an operating context, where decision-makers and operational staff have direct access to classified single-source reporting, the intelligence profession must adapt. In the age of the data deluge, intelligence managers will need to make greater use of quantitative and qualitative analytical capabilities, especially those of subject-matter experts, from economists to data scientists.
In responding to these challenges, intelligence professionals, whether they’re in the military or in law enforcement, can ill afford to view structural and organisational barriers to innovation as reasons not to try.
In his recent report, Michael Chi made it abundantly clear that there is indeed a lot of hype around big data analytics. But underneath the hype, this growing field of science offers a whole new world of capabilities and possibilities. Yes, we’ll need to develop new human resource capabilities to exploit that potential. And yes, we’ll need to find ways to fuse or integrate discreet information architecture. But neither problem is insurmountable.
The next generation of big data analytic and AI capabilities will not solve the challenge of war. But it’s not meant to. Its raison d’etre is to reduce the uncertainty of decision-making by solving puzzles and exploring mysteries. Whether developed by AI or intelligence professionals, intelligence brings no guarantees and must be weighed on a scale of probabilities.
Big data analytics and AI have already proven their worth as tools for the intelligence fraternity. Arguments to the contrary are reminiscent of the pointless debates between operational staff over the superiority of sigint or humint as a collection discipline; they are mutually reinforcing.
No comments:
Post a Comment