Big data tools improve GEOINT exploitation

John Edwards and Eve Keise

May 13, 2015 

Big data is enhancing geospatial intelligence for all of the military, including special operations forces, which rely on deep and accurate GEOINT to accomplish missions.

"With the proliferation of sensors and collectors, the volume of data available to the analyst is growing rapidly," said Lee Hall, director of exploitation and visualization services for Lockheed Martin. "To respond, you have the choice between hiring exponentially more analysts, which is impractical, or you've got to give the analysts you have the right tools to help them make sense of the flood of data they're faced with."

"Big data is forcing the GEOINT community to explore new ways of finding what information is needed within large data sets and in making correlations between sets of different types of data, whether it's wide-area motion imagery, full-motion video, hyperspectral imagery or electronic, signals or human intelligence," said John Marion, president of Logos Technologies.

Geospatial analytics applied to big data can bring to light patterns that may not be detected from other types of analysis workflows. "Big data presents a challenge to traditional GEOINT analysis in that legacy tools and databases cannot easily operate against millions or billions of records," said Brian Lehman, defense solutions director for Esri, a geographic information systems (GIS) mapping software developer.

Many large data sets are collected without analysts having any prior knowledge of what useful elements they might contain. "This data is generally not intended to be parsed and analyzed by a person," Marion said. "Tools that make correlations between data sets, such as geotemporal correlations, are very helpful."

Unique needs lead to new solutions

Special operations have unique needs for big data GEOINT. "[Special operations forces] often require real-time support to small teams executing difficult missions that are remote and may have very limited communications bandwidth," Marion said.

"[Analysts] have access to a lot of data, but turning that data into actionable intelligence is very challenging," said Lehman. One of the biggest challenges, he noted, is sending teams "an equivalent level of rich analytics that intel analysts are accustomed to using on their workstations."

The teams are also an important source of big data intelligence. "Special operations forces are solving some of the most complex and important challenges and often lead in the development of improved methods to do collection and exploitation," Marion said.

The tools to find the needle in the haystack

A variety of big data tools are useful for squeezing actionable GEOINT out of big data. Hadoop, MapReduce, Storm, Spark and ElasticSearch are just some of the many products used to create big data-driven GEOINT. "Many of the open-source projects that contribute to the core libraries for these codebases have already included spatial components or they are relatively easy to adapt to a spatial paradigm," said Tony Frazier, senior vice president of government solutions for DigitalGlobe, a geospatial content provider.

Distributed data storage frameworks, such as Hadoop's ecosystem, are useful for sharing GEOINT. "New processing and exploitation tools that refine and fuse data prior to an analyst even seeing the data are useful," said Keith Johnson, technical director of analysis and mission solutions at Lockheed Martin. "For example, change detection or object recognition analytics that process large volumes of static and motion imagery are useful to help reduce the amount of imagery an analyst needs to exploit." Also helpful, Johnson said, are "new visualization tools that present an image to an analyst and automatically identify what is already known in the image, like landmarks and previously identified targets, as well as suggesting areas for an analyst to focus on."

Visualization allows decision-makers to quickly grasp complex information, especially important to cybersecurity. "Organizations must be able to obtain situational awareness of their networks — across all data feeds — through tailored visualizations to help facilitate analysis, investigation and response," said Blake Moore, director of operations for Splunk's security markets.

According to Moore, the most useful big data GEOINT technologies take data from any source and, in real time, create analytics with visual depictions highlighting human activity on geographical overlays. "To truly create valuable GEOINT, an analyst must be able to query this information — through retrospective investigation — to show patterns, trends and anomalies in activity to discern meaning and create intelligence," Moore said.