29 April 2016

Open-source intel: NGA taps crowd for better tools


April 26, 2016 

When the National Geospatial-Intelligence Agency released eight new open-source coding projects earlier this spring, the move marked a milestone in the agency’s efforts to share the task of software development with the wider developer community.

“They are doing exceptional work,” said Brian Monheiser, director of U.S. federal programs for open-source geospatial software provider Boundless Spatial Inc. “They are sharing their expertise with a much broader constituency, putting themselves out there to collaborate with a community of people they have never worked with before.”

In 2014 NGA became the first intelligence agency to use GitHub, a publicly accessible software development network with more than 14 million users and over 35 million repositories. The agency has since launched a steady stream of projects there.

Asking questions in a big room 

In its most recent round of projects released on GitHub, NGA is seeking help with software spanning a range of topics, said Chris Rasmussen, manager of NGA’s open-source Pathfinder project. A few examples:

Storytelling: This project asks developers to construct better ways for NGA to present information in a usable format, especially when multiple authors are putting together an intelligence narrative. When working with geospatial intelligence, or geoint, “sooner or later you are going to have to convey that in English to somebody,” Rasmussen said. Better coding could make that easier to do. 

Social media picture explorer: NGA needs better tools to enable the search for specific images within pictures or video. “It’s something a 5-year-old can do: Look at something and identify it. But how do you get a machine to do that?” Rasmussen said. 
Recognizing reflections: Suppose NGA were to buy infrared imagery in the public marketplace, which is a common practice. For intelligence purposes, it would be great to know what materials are causing certain reflections on the ground. The U.S. Geological Survey has a library of reflection signatures. The open-source challenge is to connect the dots between images and library data. 

NGA increasingly has turned to open-source developers to tackle the problems of geoint. Through open-source projects, the agency can tap into nonproprietary techniques shared by a community of inquisitive developers and build its tool set while disclosing only unclassified data. In August 2015, the agency formalized its open-source efforts with the launching of the GEOINT Pathfinder project.


A screengrab of work on NGA's Social Media Picture Explorer, which allows users to explore social media by using state of the art machine learning techniques to cluster similar images and provide automatic object recognition. (Photo: NGAGEOINT GitHub)

Pathfinder pulls together a multi-disciplinary array of data scientists, application developers, open-source researchers and analysts. Participants come from NGA as well from the National System for Geospatial-Intelligence, the Allied System for Geospatial-Intelligence and developers in the GitHub community.

Driving that much talent toward open source “is a way to broaden the talent base, to truly crowd-source and to see what comes through,” Rasmussen said. The agency’s open-source projects have been especially helpful in attracting small businesses, minority-owned businesses, even graduate students: In other words, a range of developers who otherwise might not have the resources or the know-how to tender their talents in the government marketplace through traditional lines.

Some in the geoint community see the Pathfinder project as just one of several indications that NGA is pressing hard on its open-source agenda. For example, the agency has been deeply involved in LocationTech, a geospatial working group of the Eclipse Foundation.

“That’s an indication that they really are serious about open source,” said Eddie Pickle, managing director open-source programs at geoint software and services firm RadiantBlue Technologies, Inc. “It’s one thing to have an open-source project, but when you start joining industry working groups you are saying: I want the world to know about this.”

A commitment from the top

The latest release of projects to GitHub reinforces longstanding commitments made at the highest levels of NGA to engage with the open-source community.

Speaking at a gathering of industry representatives at the agency’s Virginia headquarters in spring 2015, NGA Director Robert Cardillo promised that the incorporation of unclassified input into the agency’s classified sources would create “more exquisite insights and understanding on the new, higher, open playing field.”

While much of NGA’s work will always remain classified, there is much that can be culled from the skills and experience of developers working with publicly available materials. “Classified sources, methods and networks will always have value in our agency and to our customers, but we cannot always view unclassified information as supplemental,” Cardillo said. “Moving forward the reverse is more likely to be true – that which is exquisite but classified will supplement an ever broader and richer unclassified base.”

The long-term goal is to use open source and other tools to tap into unclassified data that potentially could be valuable when meshed with NGA’s in-house trove of information. “We want to see what kind of data we can leverage to enhance our classified data, to add another layer of information to compare that against,” said John Goolgasian, NGA’s Director, Source/Content Portfolio Manager.

NGA managers believe unclassified data accrues value when it is put to use by developers seeking better solutions, especially in the open-source world. “There are a lot of people there who feel that if they each can just contribute a little bit, the quality of software becomes much stronger than it could ever be otherwise,” Monheiser said.

For NGA, “they get many sets of eyes and ears on the technology that maybe they wouldn’t otherwise have. They get some fresh insight into how those things work and how they can maybe be enhanced and improved,” he said.

The legal bits

While observers say NGA has made great strides in its move to embrace an open-source philosophy, Rasmussen said there have been challenges along the way.

Fundamentally, there’s the idea of engaging unpaid, nonemployee, noncontractor labor to tackle government IT issues with potentially sensitive data at play. It’s hard to know what to do with that.

Issues of intellectual property arise, for example. Questions about government contracting, statements of work, marketing, security, licensing analysis and the content of the project request itself must somehow be addressed. “There is the administrative work, the legal work. The time that is needed to do it is just massive,” Rasmussen said,

The agency is working to develop a plan that might streamline some of these processes. Within the next 18 months, NGA plans to stand up an office solely devoted to managing its open-source projects. This team of experts will give in-house customers a single point of contact in their efforts to bring projects to GitHub. “We really need a corps of people who do nothing but that all day,” Rasmussen said.

In a further evolution, NGA planners intend to introduce money into the picture. In the usual model, GitHub developers will take on projects for a variety of reasons. Some like the intellectual challenge. Others may hope to parlay their developments into commercial projects somewhere down the line. Now NGA is stepping up the game by offered cash rewards to those with the most promising outcomes.

Using its new “challenge authority,” NGA will offer rewards ranging from $10,000 up to $1 million to developers who can crack some of the tougher nuts. Payouts will depend on the complexity of the project.

“This is part of working transparently,” Rasmussen said. “You are contributing open-source code for everybody, and now there is a decent monetary award. [Developers] also don’t have to know the subtleties of government contracting and be able to do all that. This money is available to anybody.”

No comments: