Predicting Cyber Attacks: A Study of the Successes and Failures of the Intelligence Community

by Leslie Stanfield

July 7, 2016 

Predicting Cyber Attacks: A Study of the Successes and Failures of the Intelligence Community

Technology capability and dependency has increased at a tremendous rate in the last three decades and has changed the way we live our lives on a daily basis. Technology developments that claim origin with the original mechanical computation device of the early 1900s can now operate anything from your AC in your house to the Satellite providing direct support to an operation overseas. Everything from individual credit card transactions to our defense communications networks rely on a network of computer and communication systems to operate. When you swipe your credit card at a gas pump the data is sent through phone lines, computer systems, servers, satellites and relay stations and you as a consumer see none of this physically happen and expect it to be secure. Most of the literature on the topic of cyber security discuss how the private sector and governments alike are attempting to deal with the threat of these systems being accessed by individuals meant to do harm. These entities have been in varying degrees of reactive modes up until the last few decades. Real questions are being asked all the way down to the consumer level about security. How degraded is your privacy with the use of cyber technologies? What systems are vulnerable to a cyber attack? Could a cyber attack be used to prepare the battlefield for a kinetic attack? Solving these multifaceted questions require detailed analysis and cannot be solved by the answering of one question.

As I looked at these issues it became obvious that early detection and prevention of cyber-attack is an extension of the battlefield and thus the United States Intelligence Community has a big role to play. As networks become more complex and widely used they will inevitably be exploited by individuals, groups and state actors to conduct nefarious activities. As the technologies expand, the vulnerabilities must be identified as early as possible to counter their potential for negative use. Furthermore, the concept of cyberspace is constantly evolving into areas not originally thought to be an “easy target” and is very quickly becoming the next battleground for attacks executed by individual actors and nations alike. Just last year the head of the National Security Agency (NSA), Admiral Michael Rogers, stated that “China and “one or two” other countries have the ability to launch a cyber-attack on the U.S. power grid and other vital infrastructure.” (News VOA 2014, 1). A cyber-attack executed by a hacker or group of hackers against the power, financial or communications infrastructures would deal a devastating blow to the US economy with a few strokes of a keyboard.

A majority of the literature on cyber threats focus the vulnerabilities in today’s cyber space but how is the Intelligence Community adapting to these ever-changing technologies to give the decision makers the ability to react to prevent the attack? As the problems we face from a cyber-threat are very complex, I wanted to focus on identifying the major successes and failures of analysis from the Intelligence Community (IC) to predict cyber-attacks against the United States. The goal of this research is to break down the components of a good cyber defensive force into variables so I can clearly identify those failures and successes and their effects on the operational ability of the IC in cyberspace.

Literature Review

Since the beginning of the 21st century, there has been a growing amount of literature on Cyber Operations and the complexities associated with it. There was not however, much literature offering substantive analysis on cyber-attacks and their effectiveness. This is likely do to the fact that the concept of Cyberspace is a relatively new to the IC and that are still some gaps to be filled. There were notable patterns that developed in the research that collectively offer a better understanding of cyberspace operations. In order to understand the technical concepts in the research, a baseline of knowledge of cyberspace must be obtained. The following paragraphs will establish that baseline and subsequently focus on the areas pertaining to cyberspace that can categorize failures in cyber defense.

In order to understand why cyberspace is such a complex problem for the intelligence community a baseline of knowledge needs to be established on the components that define cyberspace. The clearest and most relevant resource for this information is contained in the US Government Joint Publication (JP) 3-12 Cyberspace Operations. Published in 2013, JP 3-12 explains the joint operational environment of cyberspace operations and defines the roles and authorities that exist in the cyber domain for the Department of Defense (U.S. Government 2013, 6). JP 3-12 and a few other sources, build the baseline knowledge of cyberspace and a basic history of technological advancements over the last decade in order to understand and plan for the technological advances to come.

To understand cyberspace as an operational environment one must first define cyberspace itself. JP 3-12 defines cyberspace as “A global domain within the information environment consisting of the interdependent networks of information technology infrastructures and resident data, including the Internet, telecommunications networks, computer systems, and embedded processors and controllers (U.S. Government 2013, 69).” Understanding that the infrastructure that is cyberspace is so expansive and critical to national defense, JP 3-12 establishes cyberspace as the fifth operational domain alongside the previous existing domains of air, land, maritime, and space (Mudrinich 2012, 172). Understanding the extensive nature of cyberspace is important as it lays the framework for understanding that the complex system of networks, computer systems, controllers, and information are vital infrastructure that are essential for the civil and military operations of most developed countries (U.S. Government 2013, 15). 

The second concept to grasp is the pace at which technology advances have been made over the past fifty years. In the decades leading up to 1980, computers were very large and expensive and were not being widely used. The large corporations that could afford their high cost primarily dedicated their computer use to high volume data processing. None of the computers at that time had microprocessors yet and primarily consisted of tubes and wiring for circuitry. It was not until the late 1980s that serious technology developments occurred in processors and the number of personal computers began to increase dramatically as the cost associated with them decreased. Catalin-Iulian Balog pointed out that the internet took only five years to reach an audience of 50 million to which radio took 35 years and television took 15 years to do (Balog 2014, 11). Internet usage and personal computing has continued to grow exponentially since the turn of the century. This growth begins to show how rapid technology advancements have paved the way for an infrastructure that is vulnerable to targeting and exploitation by our adversaries.

When looking at the IC for vulnerabilities in our cyber defense the first area of interest lies in the allocation of defense spending. According to a 2015 Stockholm International Peace Research Institute SIPRI Fact Sheet, the US Defense budget for 2014 was $610 billion followed by a distant second of $216 billion by China (Perlo-Freeman et al 2015, 1). In order for the United States to remain the world’s most capable and robust military, there must be massive expenditures within the budget to sustain it. Without this spending, the capability inside all domains would rapidly decrease and this is especially the case in the cyber domain. As the networks within the Cyber Domain become more and more complex, more spending will be required to operate and defend those networks. Without the proper funding, there would be defensive gaps that would create weaknesses in the network infrastructure. An example of this that John Cobb identified in his article was the need to better prepare our IC to defend our networks by setting up small closed networks to train them and develop tactics, techniques, and procedures (TTPs) to operate within (Cobb 2011, 81). This closed network could allow for testing intensive validation of research already available like Varun Dutt, Young-Suk Ahn, and Cleotilde Gonzalez’s article that created a model to understand the adversary’s Cyber behavior (Dutt et al 2013, 605). Their research looked “to determine the effects of an adversary’s behavior on the defender’s accurate and timely detection of the network threats (Dutt et al 2013, 605).” This application would greatly assist in building up the necessary technical skills that are required to operate in the Cyber Domain but was not the focus of any DoD entity prior to the establishment of the US Air Force Cyber Command in 2006. Without proper funding, there would be limited growth in defensive effectiveness while the adversary’s capability continued to grow.

As a direct result of the lack of funding the actual operational size of the US Cyber defense capability would be inversely proportional to the immerging threat. In the article Toward Attaining Cyber Dominance Martin Stytz and Sheila Banks identified that the nations that flourished were those that are able to “maintain strategic and tactical dominance in its critical elements of cyberspace when required (Stytz and Banks 2014, 55)”. Prior to the establishment of United States Cyber Command (USCYBERCOM) in June of 2009 there was not a unified entity that focused on the cyber security aspect of the DoD network infrastructure. John Cobb identified this vulnerability in his article when he stated that any “massive centralized network” is very vulnerable to a successful attack (Cobb 2011, 81). Cyber defense is a complex task that requires a great deal of dedicated manpower to combat the constant threat. Andrew Liaropoulos identified that the types of attacks are only limited to the “creativity and skill of the attackers behind them (Liaropoulos 2010, 181).” As the individual’s technical ability and technological capabilities increased then the more information they can exploit and the more damage they can inflict to a system (Liaropoulos 2010, 181). Without a properly manned cyber defense network the sheer number of attackers would eventually be able to overwhelm the network and meet their attack goals.

In 2011, Cobb offered an organizational model in his research that was counter to the way the Air Force operated with their cyber command at the time. His concept suggested that instead of having only that one large unit that operated cyber defense operations for the entire Air Force that there should be similar access to that defense at the base information technology (IT) level so that the bases can share in the cyber defense workload (Cobb 2011, 81). They should be able to run their own networks, have the ability to respond to network threats and the tenant units should conduct drills to simulate network outages to ensure if a successful attack does occur operability is not lost (Cobb 2011, 81). Operating under this model would provide units with an additional defense instead of relying just on an element that does not reside within the local area network (LAN). This would increase the responsibility of the base IT personnel in the form of more advanced training and the additional network responsibilities.

The ever-changing environment of cyberspace requires personnel to maintain high levels of expertise to remain effective. David Chaikin presents examples of this in his research where he focuses on locating an adversary with information that has “short life span” on servers that may be outside of the United States (Chaikin 2007, 239). Capturing this data as useable intelligence is extremely difficult and requires the employment of experts in the cyber security and network operations realm (Balog 2014, 11). Chaiken’s article focused on this complexity and concluded that the “the process of collecting, securing and analyzing digital evidence is a new forensic field where there is a lack of trained and experienced forensic investigators and examiners (Chaiken 2007, 256).” Gaining that required level of expertise is time consuming and expensive. Additionally, Varun Dutt, Young-Suk Ahn, and Cleotilde Gonzalez point out the challenge with analyzing the lessons learned in successful cyber-attacks that occur against some of the larger networks (Dutt et al 2013, 616). This is likely as a result of those attacks being on networks where the owner either does not benefit from the release of the results of the attack or the data is very sensitive in nature (Dutt et al 2013, 616). The information gap this could produce would quickly result in a vicious cycle of network penetration that stemmed from the inability to develop defense TTPs. Learning and constantly developing the IC’s cyber capabilities is increasingly important as more and more technology advances occur every day that continuously complicate cyber defense operations.

Likely, the most complex issue faced while operating in the domain of cyberspace is in how it is defined as an operational battlespace and how we are to utilize it as a battlefield. Andrew Liaropoulos describes the concept of cyberspace as an operational battlespace by using Just War Theory. In his article he defines Just War Theory as the “attempt to conceive of how the use of arms might be restrained, made more humane, and ultimately directed towards the aim of establishing lasting peace and justice (Liaropoulos 2010, 178)”. Liaropoulos splits this theory into three stages. The first is stage is “jus ad bellum” where the concept of warfare legally begins (Liaropoulos 2010, 178). The second stage is “jus in bello” where all warfare occurs and what he describes as the law of armed conflict (Liaropoulos 2010, 178). The third stage is “jus post bellum” in which the conflict transitions back to peace (Liaropoulos 2010, 178). When analyzing each of these he identified “jus ad bellum” as the most difficult to articulate in the cyber domain. When dealing with a cyber-attack, the difficulties associated with identifying the country of origin and if it were a state sponsored attack would make declaring war unclear (Liaropoulos 2010, 181). Liaropoulos was able to ascertain that international law needs to delineate what is an act of war and what is not (Liaropoulos 2010, 178). Joseph Nye, in his article From bombs to bytes: Can our nuclear history inform our cyber future? agrees with Liaropoulos but identifies the problem with the inability to implement this or even authenticate compliance due to third party actors likely not acting on behalf of any government (Nye 2013, 12). Although the literature supported a better understanding of cyberspace as a defined battlespace there are still complicated questions to answer.

Throughout the research, the literature offered up identifiable trends in cyber defense and complexities in cyber operations. My research will analyze these trends and attempt to fill the gaps in the literature so that it is possible to analyze the effectiveness of the IC to predict cyber-attacks. This will provide a better understanding of cyberspace operations and assist in identifying possible TTPs for the IC.

Methodology

Most of the literature on the topic of cyber-attacks and cyber threats focuses on the ever-changing cyber environment and does not provide much insight into the operational lessons learned for the various Intelligence Communities. This research will concentrate on analyzing the variables effecting cyber defense to determine patterns of failure and compare those with other researchers like Andrew Liaropoulos who applied Just War Theory to cyber warfare. Additionally, leadership errors like cognitive entirety will be utilized to describe the reasoning for the slow establishment of the US cyber capabilities. Since there has not been research conducted to identify patterns of failure, bias or error in decision making within the Intelligence Community relating to cyber-attacks, I will use concepts from the disciplines of Economics and Sociology to assist in answering the research question.

While conducting the initial analysis a pattern in the literature began to appear that pointed to the intelligence community not being initially prepared for the rapidly advancing cyberspace developments. The ability of the Intelligence Community to predict and construct a good defense against the cyber threat was a direct result of factors relating to both Economics and Sociology. In order to determine the reasons why these failures occurred they needed to be broken down into variables and analyzed individually so that a better understanding can be obtained of their effect on the dependent variable. The dependent variable is identified as the breach of security as a result of a cyber-attack. The independent variables are identified as; IC lack of funding, IC lack of size, IC lack of expertise and lack of senior level political emphasis. With these variables identified, I will use a mixed methods research approach focused on the Social and Economic Sciences and Intel Studies theories to articulate the successes and failures of the Intelligence Community to predict cyber-attacks. Applying the theories discussed above will identify the breakdowns in the IC and produce methods to prevent further successful attacks.

When developing the framework for this research I found some serious limitations and information gaps present in of the literature. The Intelligence Community’s task of detecting an attack and creating a sufficient defense against that cyber-attack is complex and the information gaps added further complications. Most of these gaps exist because the framework of cyberspace is relatively new and the analysts do not have access to a robust lessons learned reflection that other disciplines like Human Intelligence (HUMINT) have. As with other disciplines, the more reflection spent on the problem set the more solutions can be obtained. 

The first of these gaps in the data rests with the inability, at some level, to identify all of the intelligence successes. The nature of the cyberspace does not display intent in the same physical sense that a commander would on the battlefield. On the battlefield, you can conduct a battle damage assessment (BDA) and understand what the successes and failures are but in cyberspace that is not the case. A single unsuccessful network attack can bring down the entire network and there would be little to know traditional BDA if the attack was successfully blocked. Identifying the scope of a successful defense becomes difficult at best. Being able to say, the attack that we just defended against would have resulted in the loss of this specific intellectual property is not likely. Success in the intelligence community is more commonly described as simply the lack of detected network penetrations.

The second of these gaps was in the rapid creation of new technologies, policies, and changes in cyber operations that have not yet be represented in the literature. Technology exists in such a rapid growing field that any time spent away from the “execution on the ground” can result in not understanding an immerging threat. This gap will be filled by focusing on the open source government reporting and analyzing the trends in the information available on cyber threats in the recent past. 

The final identifiable gap is the lack of international law in the cyber domain that clearly shows the difference between an act of war and a criminal act. As the cyber domain continues to develop and becomes more understood this will be something that the international community will have to focus on in order to maintain order between all nations. Until this occurs, having a good understanding cyber conflict and the concepts of Just War Theory will provide the ability to bridge the gap of understanding. My research will attempt to fill these gaps and continue to explain how we navigate the complexities of the cyber domain.

Analysis and Findings

During the course of my research, I found that the literature was sometimes inconsistent when identifying the cyberspace strength and capabilities of the United States Intelligence Community. In order to test my hypothesis that the Intelligence Community has not been adequately prepared to predict cyber-attacks against the United States I focused on analyzing the four primary variables I mentioned above to paint a better picture of their abilities. I will show how decision-making failures effecting the Intelligence Community’s lack of funding, size, expertise, and senior level political emphasis led to the inability of the IC to execute a strong cyber-defense and whether the community has made any operational progress in cyberspace.

Senior Level Political Emphasis

The first and most important independent variable was the leadership’s inability to acknowledge the cyber threat with the level of importance that it deserved. Although the capability of hacking have been around since the mid 1980’s it did not become a focus of many United States leaders until the early 2000s. This created an environment that has put the IC behind in capability from some of the other players in the Cyber Community. The term Hacker was first introduced in a Newsweek Article in 1983 entitled Beware: Hackers at Play, which detailed the Federal Bureau of Investigation’s arrest of a group called the Milwaukee 414s (CQ Researcher 2013, 1). It was not until after the attacks on September 11, 2001 did lawmakers enact the 2002 Cyber Security Enhancement Act in an effort to solve this complex problem (CQ Researcher 2013, 1). Other than a few agencies working on cyber problem, there was no top down emphasis on cyber operations for the Intelligence Community and this held them from making advancements in the cyber domain.

It has only been in the last two decades that a leadership emphasis has been publicly placed on the issue of cyber security. It took quite some time for those decision makers to understand the security ramifications of the technology advancements and how rapidly they were intertwined in the American way of life (both civilian and military). This change of direction in the leadership came as a result of seeing the damage that can be inflicted by cyber-attacks and resulted in the United States Intelligence community being retroactive in cyber defense. General Dempsey acknowledged in a comment in 2013, that cyber security has “escalated from an issue of moderate concern to one of the most serious threats to our national security. We now live in a world of weaponized bits and bytes, where an entire country can be disrupted by the click of a mouse (Nye 2013, 8).” It is by this slow leadership reaction that individuals, groups, and state acting cyber threats around the world were given a developmental head start.

This decision-making error is explainable with the theory of Cognitive Entirety. In Martha Cottam’s article entitled Cognition, Social Identity, Emotions, and Attitudes in Political Psychology, she looks at the psychological aspect of how people process information and identifies some of the flaws that occur as a result (Cottam 2009, 55). In the cyber realm, it can be argued that there was a relative sense of security for the data being transmitted through cyber space when in reality this was not the case. This false sense of security lead to failures in the IC’s ability to detect threats to that data. This type of analysis error can stem from the bolstering that could be occurring at the political policy and analytical level. Cottam describes this as the individual selecting only the information that supports his/her decision and ignore other information that contradicts them (Cottam 2009, 55).

Once identifying the importance of cyber security, senior level political emphasis has to be placed on establishing the legalities of operating in cyberspace. Andrew Liaropoulos attempts to assist with solving this problem using Just War Theory in his article War and Ethics in Cyberspace: Cyber-Conflict and Just War Theory. In his literature he describes that Just War “has provided us with one of the most perpetual frameworks for the question of when it is right to go to war, and how war ought to be conducted” (Liaropoulos 2010, 2). He describes cyberspace using Just War theory in a way that it gives the intelligence community the ability to understand the reaching effects that uncontrolled cyber warfare can have on a nation. In the three phases, he lays the framework for the transition from peace to war, how to operate while at war, and when to transition to peace. Since there are no international laws that spell these transitions out it should be treated the same as any domain by identifying the components of cyberspace similar to that of the battlefield. Understanding this will provide a better concept to the IC on what their responsibilities and limitations are. Advances have been made in the last few years that lay out these operational constraints ranging from Executive Order 12333 and the development of Joint Publication 3-12 (R): Cyberspace Operations.

Funding

The research shows that the second, third, fourth independent variables are closely linked. The first of these is the IC’s previous lack of funding. As discussed in the literature, a country is only as great as its ability to protect itself and that ability is driven by money. This is no different for cybersecurity. Just this last year the United States cyber-operations budget was increased to $4.7 billion, up from $3.9 billion the year prior (Michaels 2013, 1). This budget increase coincides with the increase focus on cyber operations in the United States. This level of spending on cyber security has not always been the case. In 2000, President Clinton “developed and funded new initiatives to defend the nation's computer systems from cyber-attack (Chaisson 2000, 1).” Prior to this, there had been no significant allocations on funds towards programs protecting the US from cyber-attacks. This recent spending has increased the capabilities of the cyber community by creating more units that are able to focus on cyber operations.

Size

The next variable is the lack of size of the IC community in dealing with the cyber threat. As the threat continues to increase so must the size and capability of the cyber defense network. The literature has shown a considerable increase in the size of the cyber units under the Department of Defense for the last ten years. The growth of cyber units has included the Air Force establishing the US Air Force Cyber Command in 2006 and the US Army established United States Army Cyber Command in 2010. In 2009, the Department of Defense saw the need for an increase in the need of unified command for cyber operations and made the decision to create the United States Cyber Command (Gates 2009, 1). In the memo establishing USCYBERCOM, the Secretary of Defense Robert Gates recommended “the President re-designate the position of Director, National Security Agency (DIRNSA) as the Director, National Security Agency and Commander, U.S. Cyber Command as a position or importance and responsibility under the provisions of title 10, United States Code, Section 6o1 and authorize it to carry the grade of General or Admiral (Gates 2009, 1).” This was a logical partnership since the NSA was already in charge of the operational use of the National Signal Intelligence (SIGINT). The establishment of USCYBERCOM created a unified command structure within the Department of Defense that was led by the DIRNSA and created a smoother working relationship with the DoD and the NSA. 

Expertise

The growing size of the cyber community inside the Department of Defense and the unity of command under the Director of the National Security Agency positively effects the cyber expertise of the Intelligence Community. By unifying the cyber commands each of the services were empowered with the same mission and the capability to produce the best cyber professionals possible. By having all of these experts in one unit, they are able to receive continuously training on the immerging technologies and ever changing threats. In the current operating environment, this unity is important as every computer we use represents a vulnerability for the IC. The specialization in cyber operations under the reorganization of USCYBERCOM in 2009 played a pivotal role in creating an environment that greatly benefited the expertise of the IC in cyber operations.

What is Success in Cyberspace?

There have been numerous high profile cyber-attacks over the past few years that the IC was not successful in predicting or defending. The most notable of these was the Office of Personnel Management (OPM) hack that occurred in early June of 2015. This attack resulted in the OPM losing nearly 21.5 million individual personnel files of individuals containing details from security clearance investigations completed in since 2005 (C4I News 2015, 1). Although the details of the attack are still classified, the attack was likely completed by using the spear phishing concept that enticed a user on the Office of Personnel Management network to click on a malicious link to give the adversary access to the computer. A majority of cyber threats rely on that inside person to drop their guard and click on that link to give the hacker that access. Despite the fact the Department of Defense has long been training its service men and women to exercise caution when operating on the DoD networks, it still occurs. This type of end user security failure is almost impossible for the IC to prevent without incredibly reducing the internet capability. 

Based on a cursory look at the literature it appeared as if the IC was not sufficiently resourced or staffed to conduct successful cyber operations. The problem in identifying success in the cyber world can be seen only as an attack on a critical node that never materialized. Even if one could identify the attack on the network, it would be difficult to know the extent of the damage of the attack. This is mostly due to the classified nature of Department of Defense and the National Security Agencies operations in cyberspace. I found that the IC has made incredible advances towards improving our cyber-defense capabilities in the complex environment that is cyberspace, but this was not the case prior to the year 2000. At that time, the IC was in no way prepared to take on the cyber threat and it took senior leadership emphasis to get the IC on track to where it is today. Had this change not occurred the United States IC would not be operating with the level of capabilities that it is today.

After thoroughly analyzing the information available in the literature, I am able to conclude that my hypothesis, the Intelligence Community has been unable to predict cyber-attacks against the United States, is false. The IC’s operational ability to operate in the joint environment of cyberspace has greatly increased after the establishment of USYBERCOM in 2009. The senior level political emphasis on cybersecurity coupled with the increase in funding, the creation of additional units and increase in cyberspace operational knowledge has created a positive way ahead for the cyber community.

Conclusion

This research compared the cyber successes in Intelligence Community with concepts from the disciplines of Economics and Sociology to assist in answering the research question. As our society relies more and more on technology for their day-to-day lives, the risk of that information being exploited grows exponentially. As the risk increases so does the level of responsibility of the IC to ensure we are protected from and adapt to the threats of the future. The operational capabilities of the IC’s cyber operations has increased in the past decade and as a result of the leadership understanding the importance of cyber defense the US now has a pivotal role in the cyber community. Those leaders identified that our capability in both numbers and training needed to grow to continue to be successful on the battlefield. 

Even with an increase in focus on cyber security, there are still attacks we are unable to defend against. This could be for a number of reasons ranging from the enemies capability increasing or a failure in our ability to adapt and grow in the cyber domain. In most cases we have no way of identifying what attacks we are unable to defend against as they will not be released as public knowledge unless it directly effects our citizens. An example of this would be in the OPM hack. USCYBERCOM and the NSA will likely never release the way the hackers accessed the system or the vulnerabilities that existed to allow the intruders in. Since this is classified information it is likely that it will not ever be released to the public. A possible solution to this could be in use of research conducted at the classified level that would develop the lessons learned for the cyber community to share.

One major focus of continued research needs to be in the area of more clearly defining the operational domain of cyberspace. Unlike the clear operating environments in the domains of Air, Land, Maritime, and Space, there are many questions that still exist as a result of having no operational boundaries in cyberspace. Some of those questions include “Who determines the rules of agreement in cyberspace?”, “What makes an action in cyberspace an offensive action versus a defensive action?”, “Does a good defense rely upon a good offense in cyberspace?”, and “Are there country boundaries that if crossed could be considered an act of war?” Providing answers to these questions and defining boundaries of cyberspace will require additional time for the international community to develop the necessary tactics techniques and procedures to analyze. Research focusing on previous incidents of cyber-attacks to begin the process of identifying those boundaries will be helpful in understanding the complexities of cyberspace. Thinking outside the box and using multiple disciplines to assess the complex problems associated with cyberspace operations.

Cyber-attacks are going to continue to be a threat to anyone who uses technology. Research in the future needs to continue to develop ways to combat the cyber threat against the ever-changing technologies in our society. As more and more technologies are developed, there will be more and more opportunities for individuals to conduct future research into how those technologies will effect operations in the cyber domain.

References

Anonymous. 2010. "IT SECURITY: Open to cyber attack." Process Engineering: 26-n/a,http://search.proquest.com/docview/213864382?accountid=8289. (Accessed July 18, 2015).

Balog, Catalin-Iulian. 2014. "NATIONAL POLICIES AND STRATEGIES ON CYBER SECURITY." Carol I National Defense University. http://search.proquest.com/docview/1528149408?accountid=8289. (Accessed July 18, 2015).

C4I News. 2015. "OPM Director Resigns Amid Cyber Breaches." C4I News,http://search.proquest.com/docview/1696933762?accountid=8289. (Accessed July 19, 2015).

Chaikin, David. 2006. "Network investigations of cyber attacks: the limits of digital evidence." Crime, Law and Social Change 46, no. 4-5: 239-256, http://search.proquest.com/docview/216164783?accountid=8289. (Accessed July 18, 2015).

Chaisson, Kernan. 2000. "High-level, national cyber-terrorism plan announced.". Journal of Electronic Defense 23, no. 3: 15, http://search.proquest.com/docview/206600682?accountid=8289. (Accessed August 22, 2015)

Cobb, John, U.S.A.F. 2011. "Centralized Execution, Decentralized Chaos: How the Air Force Is Poised to Lose a Cyber War.". Air & Space Power Journal 25, no. 2: 81-86,http://search.proquest.com/docview/918791788?accountid=8289. (Accessed July 18, 2015).

Cottam, Martha L., Dietz-Uhler, Beth, and Mastors, Elena. 2009. “Introduction to Political Psychology (2nd Edition).” London, GBR: Psychology Press,http://search.proquest.com/docview/199658706?accountid=8289. (Accessed July 9, 2015).

CQ Researcher, 2013. "Improving Cybersecurity: BACKGROUND." CQ Researcher 23, no. 7: 166-171. Academic Search Premier, EBSCOhost (Accessed August 22, 2015).

Díaz, Gustavo. 2005. "METHODOLOGICAL APPROACHES TO THE CONCEPT OF INTELLIGENCE FAILURE." UNISCI Discussion Papers, no. 7: 1-16,http://search.proquest.com/docview/224068538?accountid=8289. (Accessed July 18, 2015).

Dutt, Varun, Young-Suk Ahn, and Cleotilde Gonzalez. 2013. "Cyber Situation Awareness: Modeling Detection of Cyber Attacks With Instance-Based Learning Theory." Human Factors 55, no. 3: 605-618. Academic Search Premier, EBSCOhost (Accessed August 8, 2015).

Gates, Robert M. 2009. “Establishment of a Subordinate Unified U.S. Cyber Command Under U.S. Strategic Command for Military Cyberspace Operations.” Department of Defense Memorandum.https://publicintelligence.net/establishment-of-a-subordinate-unified-u-s-cyber-command/. (Accessed August 22, 2015).

Jim, Michaels, and TODAY USA. n.d. 2013. "Military devotes more money to cyber-attack capabilities." USA Today, n.d. Academic Search Premier, EBSCOhost (Accessed August 22, 2015).

Liaropoulos, Andrew. 2010. "War and Ethics in Cyberspace: Cyber-Conflict and Just War Theory." Academic Conferences International Limited. 07.http://search.proquest.com/docview/869506998?accountid=8289. (Accessed July 18, 2015).

Moore, David T. 2007. “Critical Thinking and Intelligence Analysis.” Occasional Paper No. 14. National Defense Intelligence College. https://edge.apus.edu/portal/site/299249/page/f37d2dd9-dd12-4230-b42b-23daa52a8c16 (Accessed July 15, 2015).

Mudrinich, Erik M. 2012. "CYBER 3.0: THE DEPARTMENT OF DEFENSE STRATEGY FOR OPERATING IN CYBERSPACE AND THE ATTRIBUTION PROBLEM.". The Air Force Law Review 68: 167-206, http://search.proquest.com/docview/1020878866?accountid=8289. (Accessed July 18, 2015).

News, VOA. 2014. “NSA Chief: Other Countries Capable of Crippling Cyber Attack on US Power Grid.” Lanham: Federal Information & News Dispatch, Inc.http://search.proquest.com/docview/1626699143?accountid=8289. (Accessed July 18, 2015).

Nye, Joseph S. 2013. "From bombs to bytes: Can our nuclear history inform our cyber future?." Bulletin of the Atomic Scientists 69, no. 5: 8-14. Academic Search Premier, EBSCOhost (Accessed August 8, 2015).

Perlo-Freeman, Sam, Aude Fleurant, Pieter D. Wezeman and Siemon T. Wezeman. 2015. “Trends in World Military Ependature, 2014.” Stockholm International Peace Research Institute: SIPRI Fact Sheet. http://books.sipri.org/files/FS/SIPRIFS1504.pdf. (Accessed August 8, 2015).

Stytz, Martin R. and Sheila B. Banks. 2014. "Toward Attaining Cyber Dominance." Strategic Studies Quarterly 8, no. 1: 55-87, http://search.proquest.com/docview/1516145464?accountid=8289. (Accessed July 18, 2015).

U.S. Government. 2009. “A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis.” Center for the Study of Intelligence. Washington, D.C.: U.S. Central Intelligence Agency. https://edge.apus.edu/access/content/group/security-and-global-studies-common/Intelligence%20Studies/INTL506/Tradecraft%20Primer-apr09%20US%20GOVT.pdf (Accessed July 15, 2015).

U.S. Government. 2013. “Joint Publication 3-12 (R): Cyberspace Operations.” Office of the Joint Staff Director of Operations. Washington, D.C.: Department of Defense. www.dtic.mil/doctrine/new_pubs/jp3_12R.pdf. (Accessed August 8, 2015).

Richards J. Heuer Jr. 1999. “Psychology of Intelligence Analysis.” Center for the Study of Intelligence, (June 2013): https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/psychology-of-intelligence-analysis/index.html. (Accessed July 9, 2015).