Avoiding the Secrecy Trap in Open Source Intelligence

CHRIS RASMUSSEN

Source Link

The Secrecy Problem

Open-Source Intelligence (OSINT) operating concepts in the Intelligence Community (IC) are outdated. While the IC has long cited open source in classified products, treating unclassified information as just another “INT” feeding classified systems is an inadequate model with the future datasphere approaching hundreds of zettabytes and where the most valuable data and analytic technology is coming from outside classified facilities. The IC’s meager and decades-long OSINT reforms and under-performance are symptoms of a culture designed to protect secrets. While secrets and protecting them will always be a part of a healthy intelligence apparatus, the policy, resourcing, and information technology (IT) priorities of classified operations are incompatible with a world flooded with open and commercial data and cannot scale OSINT toward a cohesive national-level mission.

Georgetown University researchers estimate that China has 100,000 open source analysts extracting value from scientific and technical developments globally but with an emphasis on the United States. China has a larger labor pool with a more government-directed interwoven labor model and scouring scientific and technical journals is a sub-set discipline of OSINT, but the 100,000 figure in this sub-set alone is still a staggering line of effort. By comparison, the IC’s fragmented OSINT efforts, of any sub-set or model, are orders of magnitude smaller with the general model of small-to-modest-sized full-time OSINT teams embedded within substantially larger classified shops. Moreover, the IC often reduces OSINT to a collection discipline to feed “requests for information” into classified products re-hosted on air-gapped and classified networks. This “highside” model where OSINT is sucked up for “fusion” is a narrow, outdated mindset and limits broad customer, partner, and Allied sharing. As evidenced by the increased demand for quality OSINT from the conflict in Ukraine, broad Allied and partner sharing is required to counter malign influence, spoil operations of our adversaries, and shame criminals masquerading as world leaders.

In 2021, defense intelligence officials testified before the House Armed Services Committee’s Subcommittee on Intelligence and Special Operations that defense and military intelligence components were struggling to respond to counter China and Russia’s false and malign online narratives. They suggested that force structure, personnel, resources, and policies were not maximized to counter internet-scale information operations. According to Ruben Gallego, former chairman (now ranking member) of the subcommittee, defense and military intelligence components have been slow to respond and opportunities have been missed to counter narratives during normal operations with the exception of combat support situations where the sense of urgency is greater. I argue the slow response during “normal” bureaucratic operations is systematic partially because OSINT and the sub-discipline social media intelligence (SOCMINT) are often bound within classified frameworks where flexibility is limited. OSINT must be set free from classified frameworks to grow and flow within unclassified and open ecosystems. This will increase its reach to policymakers, partners, and new customers, such as mobilizing online coalitions to counter malign influence and operations. The battlespace has shifted more into the open and viewing OSINT as just another part of a classified product is an outdated view that bounds open data creativity and dissemination within the organizational box of systems designed to protect secrets.

Moving up into 2022, the IC’s declassified exemplars of military staging maps of Russian equipment shared with the Washington Post prior to the invasion of Ukraine rallied the international community against Russian aggression. This is a step in the right direction, but this type of content needs to be created daily, not as a one-off. This declassified example also begs the question, did this need to be declassified in the first place? We must move to a model where quality OSINT is created from the ground up unclassified and never becomes classified for broader policymaker, Allied, and partner engagement delivered outside of classified facilities around the clock to mobile devices for internal usage or publicly to rally coalitions.

It’s not just for the President anymore. Are you getting your daily national security briefing? Subscriber+Members have exclusive access to the Open Source Collection Daily Brief that keeps you up to date on global events impacting national security. It pays to be a Subscriber+Member.

A New Model to Scale OSINT

There is a growing number of journal articles, press reports, and congressional actions with calls for OSINT reform. Some of the articles suggest it is time to form a new OSINT Agency within the IC or other revised IC functions. If allowed, this would be an error of sizable proportions. Not only would nesting it inside the IC fall far short of what our nation, Allies, and coalitions need, but it would also perpetuate a counter-productive culture of needless secrecy. From a systems thinking and organizational theory perspective, it is clear that we must create a new independent OSINT agency outside the IC. The new entity must be built from the ground up, not to protect secrets, but to create quality and shareable OSINT at scale with sufficient full-time OSINT practitioners in large numbers enabled by remote and hybrid telework, limited security clearances, and the rapid growth of the OSINT knowledge base at the unclassified level for broad dissemination, not sucked to the highside as a supplement to classified operations.

Stop Admiring the Secrecy Problem

Outside studies and calls for the IC to grade itself regarding open source have concluded similar things tracking back to the late 80s and early 90s. The common conclusion? The IC did not adequately integrate open source into classified operations and prioritized classified [information] over open data. To be clear, this is not the IC’s “fault”. These studies have just pointed out that the IC was simply not designed to function effectively in the open domain.

The U.S. Army confirmed this finding at the beginning of the internet era when they foresaw a “data explosion.” In 1998, Army Intelligence commissioned a study on improving OSINT operations titled “Intelligence XXI Task Force for Open Source Information.” For historical context, Bill Clinton was president, The Big Lebowski was in theaters, and the smartphone was eight years away from being invented. The themes brought up in the 1998 Army study are echoed in the more recent OSINT articles and studies following it: more technical talent is needed, should be the INT of first resort, data volume is overwhelming, more laptops and unclassified IT infrastructure is required, mindset is more important than buying tools, etc. This Army study is just one of many, some of which resulted in actions, but most have produced marginal reforms because they are anchored to a system where the entire operating system is linked to protecting secrets. We cannot continue to graft open data analysis to the world of secret protection and expect different results.

Open, Not Classified

The IC was designed to steal and protect secrets. It recruits people, builds systems of record, and processes which suit that mission. While there is a place for this function within government, it is wholly inappropriate to ask people who are charged with maintaining secrecy to now expose their practices to the world. It is not who they hired, it is simply not congruent with their data review and release processes (e.g., Prepublication Review), and it takes the IC away from their primary mission of acquiring, securing, and communicating classified information to policymakers.

For any open source center of excellence or new OSINT agency to succeed it must not be placed within a system where security clearances are required for employment. When security clearances are required, the first operating principle that filters down into every nook and cranny of operations is the protection of secrets: the buildings, IT systems, content creation practices, hiring policies, retention policies, most contracts (it’s called getting “tickets” for a reason), and even the design of the cafeteria revolves around getting and maintaining clearances to protect secrets. This new OSINT agency must be independent of the IC and use no more than “public trust clearances” or criminal background checks akin to the private sector. Capping the number of Secret and Top Secret clearances legislatively to a handful of liaison positions at the new agency would mitigate clearance “creep,” building Sensitive Compartmented Information Facilities (SCIFs), and subordinating OSINT to existing classified operations.

Public trust clearances or other non-sensitive screening is a common hiring practice at places like the Department of Agriculture or the Federal Deposit Insurance Corporation (FDIC). This business practice would easily transfer to the new OSINT agency and satisfy the background check appropriate for government service. By removing the SCIF anchor, this new agency could hire and retain talent anywhere in the country through remote telework. There would be no classified IT systems and all development, data conditioning, and original content creation would happen on unclassified networks.

As of the publication of this article, zero IC agencies are offering remote, public trust government positions, according to searches of USA Jobs postings. The small number of public trust clearances in the IC is confined to light contract support functions, not core government personnel. IC personnel are required to get Top Secret clearances by default and often have limited telework options. Spreading open source analyst and open data scientist roles across the country invests taxpayer dollars in local economies and complicates adversary operations by decentralizing personnel.

Overclassification

While overclassification is nothing new, the “clearance culture” is being abused to maintain employee livelihood by protecting “secrets.” For example, I heard an interesting story from some foreign service officers about the Iranian Revolution in 1979 and its connection to overclassification. The story goes that in 1979, motor pool order forms seized by the Iranians were stamped “SECRET.” This may seem comical to the educated outsider but from a simple cost-benefit perspective for someone holding a clearance, there are incentives to over-classify. Penalties only exist in one direction for under-classification. Hence, car part order forms inherit classified markings. This story may have morphed over the years, but anyone who has held or holds a security clearance knows the government classifies information too cavalierly.

Another example of where a “cleared culture” overclassifies data is with the concept known as “classification by aggregation.” This is the notion that merging several pieces of unclassified data together can make it classified. In my experience, the more unclassified data and methods used, the more unclassified something becomes; not the other way around.

To be clear, I’m not arguing that open data should not be protected. Rather, a shift is required to a new approach to handling open data. The protection guide for this independent agency should model patent and trade secret protections by asking questions such as: is this state of the art and what parts are worth protecting? Is it abstract or transformative? Is this software, algorithm, or method “novel” and “non-obvious?” The “non-obvious” or “novel” or “transformative” tests are used in patent cases and can be used as a model rather than relying on traditional classification guides as we know them. For example, mashing up AIS data (shipping and navigation data), tweets, and commercial imagery is not classified aggregation. The sources are unclassified and mashing up data is not a method that is worthy of trade secret-like protection because it is a common and well-known method. In order to get off the hamster wheel that repeats the same outcomes of overclassification, removing the underlying system of classification as we currently know it from the operations and culture of the new OSINT agency is a fresh perspective.

On the IT side, an independent open source agency should not use air-gapped systems. With a strong security mindset, you can secure and protect data in the cloud or “bare metal” without transferring it to the “highside” (classified IT system). You can two-factor, “zero trust,” and add Artificial Intelligence-based bot defenses to unclassified systems and keep things secure. As described above, the highest marking at the new agency would be “trade protected” and would stay within secured, unclassified IT, not publicly available to guard against the group under study changing their behavior, thus limiting further OSINT analysis. OSINT is not always synonymous with public analytic outputs. You can protect OSINT without telegraphing your “trade protected” research methods to the world by simply parsing access within unclassified IT systems. There is little or no need to classify this type of OSINT as “SECRET” or to send this content to air-gapped classified networks.

Cipher Brief Subscriber+Members enjoy unlimited access to Cipher Brief content, including analysis with experts, private virtual briefings with experts, the M-F Open Source Report and the weekly Dead Drop – an insider look at the latest gossip in the national security space. It pays to be a Subscriber+Member. Upgrade your access today.

Mastering Open Data as a Strategic Imperative

Invest in an Open Data-Centric Future

In the IC, unclassified investments generally lose to classified dollars from a resource perspective. Classified operations are viewed as “core business” so anything new or R&D-like suffers from the dilemma of investing in the core today or the edge tomorrow. Moreover, the IC embeds its OSINT teams within substantially larger classified operations and often reduces OSINT as a source of last resort and, therefore, the last funded. The classified core is becoming increasingly irrelevant but sunken cost fallacies and cultural inertia overstate its importance internally. The only way to break free of this budget subordination and classified-first resource mentality is independence or removing protecting secrets as a condition of employment. An independent agency’s entire top-line budget would be OSINT or unclassified operations which would reduce anchoring or subordinating OSINT under classified or other line items.

In addition to the Congressional appropriations to fund the new OSINT agency, the new agency could also set up a “working capital fund” where it could sell products, services, translations, or licensed inventions like the State Department sells passports or the Post Office sells stamps. The demand for high-quality OSINT has dramatically increased and it is time to think differently. There are some legal issues to overcome around creating a working capital fund based on content created by a government agency, but the point is to think differently and start fresh not anchoring to past mindsets. Since a legislative response is required to create this new agency, new “authorities” and operating principles can be baked in from the start rather than anchoring to outdated assumptions from the National Security Act of 1947 where much of the IC was born.

Creation of Tactical, Strategic, and Public Output Departments

While experimentation will be required to find this new agency’s flow, fit, and feel, I believe the creation of tactical, strategic, and public content as three separate departments within the agency will be necessary from the founding. Mixing tactical OSINT, strategic OSINT, and public output OSINT into one department would create an effect similar to the mushed function of a “spork”–it does both jobs of the spoon and the fork, and poorly at that–when it is best to just keep things separate for maximum impact. From an end-user design perspective, tactical OSINT, such as what unit is hiding behind that hill, often takes on more of a situational awareness, blinking dashboard design. Whereas strategic OSINT focuses on topics such as the status of hydro investments in China. Strategic OSINT often takes on more of a long-form article consumption end-user design experience. Mixing these two together would dilute both and create a “spork” organizational effect trying to do too much. It is best just to keep the labor and consumer iterative design aspects separate. Public OSINT outputs for “shame and blame” campaigns or other information operations are a discipline within itself. For example, it is crucial to understand the nuances of Instagram such as why creating “verticals” (narrow videos designed for mobile device interaction) amplifies engagement. This type of marketing skill is unique and constantly evolving and distinct from internal or “behind the firewall” tactical and strategic OSINT creation.

How the IC Would Interact with the New Open Source Agency

The laudable goal of OSINT and classified “fusion” can be achieved in a different way by leaving OSINT at its origin as an independent base of knowledge. The IC would be a consumer of the data and services produced by the new OSINT agency. The IC would not direct or manage OSINT agency employees, shape security policy, control electron flow (IT), or mandate unclassified data be re-hosted on air-gapped systems. If the IC wishes to “fuse” content and data created by the new agency, the IC can invest its own budget in “cross domain,” “reach down,” or “look down” technologies to integrate the content on higher Secret and Top Secret networks. The new agency would keep expenditures focused on retaining top talent and doing quality unclassified analysis, not trying to re-host unclassified data on other IT systems to fit cultural norms of working on the highest classified network possible.

A few liaison positions could be created between the new agency and the IC to brief the new open source analysis at classified policymaker meetings. These cleared positions should be capped legislatively to avoid “SCIF creep” or “classified first” mindsets. The IC liaisons could also submit “requirements” to the new OSINT agency, but these requirements must be stripped or abstracted to the unclassified level. The IC is fully capable of working classified requirements. Keeping the “requirements” unclassified avoids overclassification, SCIF creep, unnecessary clearance costs, and generally repeating the same old workflow habits that keep OSINT from its full potential.

This IT and clearance separation model sharpens impact and efficiency by focusing on differentiation and original intent. Stealing secrets will always have value but sometimes things need to be separated to avoid trying to do too much. You cannot expect Dunkin Donuts to start making pizza well. All of Dunkin’s systems and processes are designed to make coffee and donuts. This is similar to current IC systems and processes: they are designed to protect and steal secrets, not build knowledge from open data that does not require clandestine means.

Often fusing things together creates a less effective whole. The separation of unique functions can increase utility. OSINT and classified “fusion” will be more impactful when separated organizationally.

Stop Rehosting the Internet on Air-gapped Systems

Rehosting the internet, private clouds, and commercial imagery stores on air-gapped systems is unsustainable as data volume increases in zettabyte ranges. The IC’s cultural and operational desire to bring open and commercial data to the highest IT system of classification remains the dominant operational mindset. Open and commercial data should remain at the level it was created. Despite broad logical agreement within the IC on the statements above regarding data creation at origin, OSINT has yet to achieve “INT of first resort” status because the concept of classified “fusion” subordinates OSINT to classified operations. Blending OSINT and classified information to form a more holistic picture is a noble end state, but this formula breaks down in the implementation stage. If the SCIF or highest IT system is the “core” of business logic, then anything brought to it by data transfer is the supplement. To break the cycle, OSINT must stand alone as its own “base” and stay hosted where the information was created – at the unclassified level. Growing the OSINT base of operations at the unclassified IT level broadens Allied sharing, appropriately prioritizes IT dollars (not happening now), and grows partnership opportunities with the private sector, academia, and nonprofits.

Existing OSINT Authorities

Existing intelligence agencies would retain OSINT authorities for organic in-house intelligence support. One of the many goals of the new independent OSINT agency is to flip the script to get the OSINT numbers up well beyond small OSINT units embedded within substantially larger classified shops, which is the standard model at this time. A fresh and sizable OSINT base of separate and unclassified-only practitioners would raise the tide for all working and demanding OSINT to include fusion and consumption.

Mobile, Official Content Creation

No single IC element is doing official, unclassified-only content creation at scale with its own research labor delivered via mobile apps or secure websites “lowside” (IC lingo for unclassified networks). I define official production as content with an agency logo on it; the coordinated position, not “lesser” caveated reporting or mere access to raw open source content. In fact, most unclassified “production” gains during COVID have been rolled back and operations on classified networks have returned as the dominant workflow. The “lowside,” once again, post-COVID is the eternal supplement to classified operations rather than the “INT of first resort,” which is a declaration in name only. I believe an organization with an independent budget and substantial dedicated personnel generating OSINT content around the clock and serving it up via mobile apps or secure websites would be a run-away success. Instantly hundreds of customer types including senior officials seeking counters to malign influence could check their phones for information relevant to their daily decision cycles. Do you really need a daily “briefer” if you can deliver quality mobile OSINT content around the clock enhanced with live in-app chat functions where the customer can ask questions to the authors about the content? Leaders and customers of every type are doing this themselves in their personal lives daily checking their phones for information. This model would also save senior executives and customers time by reading and asking questions at their own pace outside of standard business hours or international time differences.

There are only proxies of mobile delivery success in the IC because something of this scale and independence has never been done before. For example, the IC-based Tearline Project is a program that delivers OSINT to mobile devices publicly in partnership with academia, think tanks, and non-governmental organizations. Tearline content is frequently cited within OSINT practitioner communities online and within national security and foreign policy reporting communities. But, Tearline is using external labor mostly from college students because no IC element has yet to invest substantially its own research labor and unclassified IT dollars to create mobile OSINT content that stays “lowside” at scale. There are no URLs containing IC agency-derived names behind login credentials doing OSINT at scale and no apps in the Apple Store or Google Play listed from any major IC element doing OSINT at scale daily as a core mission beyond one-off cases. By comparison, in terms of size and scale, Bellingcat is a successful investigative OSINT shop with a modest budget and staff. Imagine what a “small” new government agency with thousands of employees, steeped in research methodologies similar to Bellingcat, creating OSINT around the clock could do for a national-level mission.

This new agency will find its way. Experimentation will be necessary like with any growth venture, but the unmet demand rooted in the proxy examples of Tearline, Bellingcat, and any Ukraine conflict tracking dashboard is clearly evident. Something truly new is always held to higher standards than the status quo, which was also, at one point, formed through trial and error. This new agency, and even the proposal, will fall victim to a frequent trap within real innovation circles (not theater) when creating something truly original – critics demanding every detail and metric up front. This has been called the “Athena Trap” where if fully formed ideas are not sprung from the head of Zeus, with every permutation worked out in advance it shows “holes” in the idea from critics. These are not “holes” per se but the cost of doing things that have never been done before.

The Data Warehouse View is Incomplete

There is a fairly widely held view in many IC acquisition and commercial data circles that OSINT data and services can be purchased wholesale from the private sector with the intent of making it a federated library-like service. Library functions on steroids would be a core part of this new agency but acquisition and federation are incomplete OSINT constructs. OSINT should be viewed as more than a collection discipline. In addition to library functions, government officers “at the library” must also create neutral and objective products, data, and original content. The output or production side of the new agency would help mitigate reducing OSINT to current IC collection shop or data warehouse constructs.

Buying tools, subscriptions, or even artificial intelligence and machine learning (AI/ML) data mining and alerting systems does not constitute a holistic OSINT program. Full-time personnel, investing their time and talent to understand the data and analytics flowing through the tools, are vital to the agency’s success. Authoring original products explaining the findings from the tools to educated outsiders and the largest audience, not just the classified audience, is a healthy and holistic OSINT program because it combines purchasing, training, time, and original content creation. One cannot “outsource” or buy all content creation: One must also create in parallel. All three departments proposed above (tactical, strategic, and public) would holistically execute an OSINT program.

Finally, in addition to the content creation departments described above, a privacy and civil liberties function would be necessary. This function should exist “above” the daily flow and be embedded within the duties of the governing board discussed below.

An Independent Agency Governed by a Board

To avoid being “captured” by IC department heads and secrecy and clearance culture, the new OSINT agency should utilize an independent board modeled on the board that governs the National Science Foundation (NSF). The OSINT Agency probably would not need 24 board members like the NSF, but the board model helps as a “capture” check much like a corporate spin-off that remains independent by not bounding the new thing to existing or legacy headquarter operations.

A reasonable Washington, D.C. power politics observation is that an independent OSINT agency not anchored in the IC under associated traditional cabinet secretaries may lack a seat or access to the National Security Council (NSC) or President. New OSINT agency proposal legislation could model itself on prior legislation at military or intelligence reform like the 1986 Goldwater-Nichols Act or the 2004 Intelligence Reform and Terrorism Prevention Act to create a single empowered director to “speak” to the NSC and President. However, in a world awash in open data and mobile devices, the unified director speaking and seeking “rug time” (in the Oval Office or other seats of power) as a model carries some outdated operational assumptions. While briefers and voice-tracking written content will always carry some advantages, the current briefing model is based on talking to classified and potentially overclassified briefing books, not a world of daily pocket smartphone checking. From an information flow and usage perspective, a Border Patrol or social media officer at Radio Free Europe/Radio Liberty often has more need for useable OSINT than executive usage with citations in the President’s Daily Brief (PDB). The digital revolution and open data explosion suggest a relook at previous metrics and operational assumptions (White House as the ultimate customer) is needed in relation to where OSINT stands in the flow of important work done today and into the future.

Mobile content delivery at scale changes previous assumptions and makes this effort unique. Broad smartphone availability has transformed almost every aspect of daily life and has changed most industries. To argue that broad OSINT mobile delivery will have a limited impact on previous IC assumptions about workflow and customer expectations defies logic when compared to other domain transformations. This reasonable domain transfer shift should trigger thinking beyond classified briefing books. The PDB was ported into a hardened, stand-alone iPad during the Obama years but this is not a fully functioning iPad calling live services. Data and content flowing from the new OSINT Agency would be live and secure in a fully functional app. The President, PDB briefer, Border Guard, and social media officer at Radio Free Europe could all have secure OSINT app accounts with login credentials similar to banking apps with biometrics. After the President or staff checks their OSINT app, the IC can hand over the “spooky” iPad full of stolen secrets to build on the OSINT content. This is an example of manual fusion without cross domain technologies. This example is used for illustrative purposes, not to suggest that presidential workflow is the ultimate target and pinnacle metric of the new OSINT Agency. The proliferation of open data and easy-to-use technology is also diversifying customer bases throughout the “rank and file” levels of the US government and our partners both international and public seeking quality OSINT.

One cannot predict all the customers of OSINT upfront, especially when rallying coalitions of overlapping interests globally. An important lesson from the Ukraine conflict is how would one rally a global coalition supporting Ukraine if one relied on just classified information circulated to the same traditional US government policymaker customers. New customers outside the traditional US policymaker and Commonwealth Allied orbit needed quality OSINT to rally behind Ukraine. What the new independent OSINT Agency can do is be adjacent to these massive networks of new customers on the unclassified mobile side and on the public output side to spring into action and scale with official OSINT analysis, which is distinct from corporate or private OSINT. Quality OSINT of any stripe has value but there is also value in the sterile and transactional voice of government-based OSINT. This cannot be done well by peeking down to the unclassified world from a SCIF. OSINT at scale must be separated to be liberated.

Bottom line

There is broad sentiment regaining steam right now around open source and “new” functions even though the contours of these arguments have been around for a while. Many of the ideas expressed (old and new) are good ones but miss the systems thinking mark by anchoring them to processes designed to protect secrets. OSINT done at scale cannot co-exist embedded within cultures of secrecy. The “just another INT” for usage in the classified products model has reached a point of diminishing returns: there are few new customers or innovations left to achieve. A truly fresh start would involve removing the underlying operating and incentive structure from the equation by using public trust employees, recruiting talent anywhere with a remote and hybrid telework culture born from the start, capping the number of clearances to a few liaison positions, creating original content that stays “lowside” reaching new customers, and achieves “fusion” without re-hosting and subordinating OSINT to classified operations.