‘It’s going to take a lot of digging’: The Pentagon’s long search to see if anyone’s hiding in its networks

Andrew Eversden and Mark Pomerleau

Source Link

WASHINGTON – The military and intelligence community is scrambling to conduct a daunting hunt across disconnected networks to assess potential damage from an extensive federal cybersecurity breach by suspected Russian hackers.

As it searches for lurkers, one complicating factor is that the cybersecurity arm of the Department of Homeland Security warned Thursday that hackers used other means to access government and business networks beyond a software platform from contractor SolarWinds, used by the Pentagon, the military and intelligence offices. That network management platform was “not the only initial infection vector,” the Cybersecurity and Infrastructure Security Agency alert said.

The adversary was patient, well-resourced and used advanced techniques to mask its command-and-control communications, the agency said. All of those traits make crews’ search for damage or proof of a breach that much more difficult, officials told C4ISRNET.

No sign had emerged yet to indicate that the hackers had compromised the Pentagon’s unclassified or classified networks, but U.S. Cyber Command previously told C4ISRNET that the government’s most advanced cyber threat hunters stood ready for a rapid response if a breach is found.

“We continue to assess our DOD Information Networks for indicators of compromise and take targeted actions to protect our systems beyond the defensive measures we employ each day. To date, we have no evidence of compromise of the DODIN,” said Vice Adm. Nancy Norton, director of the Defense Information Systems Agency and commander of Joint Force Headquarters-DoDIN. “We will continue to work with the whole-of-government effort to mitigate cyber threats to the nation.”

The breach highlights security risks posed by third-party vendors.

Andrew Eversden and Mark Pomerleau

If hackers breached defense or intelligence IT systems, a big question investigators will face is how they maneuvered inside sensitive computer networks. Are they simply able to observe communications, a common espionage practice? Or did they penetrate systems so thoroughly that they could attack to disrupt operations or destroy systems and information?

It’s too early to know the full extent of the damage or access that the suspected hackers — the Russian foreign intelligence agency known as SVR — had in U.S. networks, according to cyber and IT professionals who previously worked in the intelligence community and military.

Several experts told C4ISRNET that they expect it to be quite some time before officials have a clearer picture of any damage.

“I couldn’t estimate on that, honestly, because ... all the DoD networks are very complicated,” said retired Rear Adm. Danelle Barrett, former deputy chief information officer of the Navy. “You’ve got a combination of legacy networks and more modern networks. Some things are more automated than others on those networks. So it’s going to take a lot of digging ... There’s probably things that they can identify right away. It’ll be awhile before they have the whole complete picture.”

The response from cyber defenders

The breach, which reached the Treasury, Commerce, Homeland Security, State and Energy Departments, kicked off a response requiring coordination across the federal government.

Leading the federal-wide effort is the newly formed Cyber Unified Coordination Group, made up of the FBI, the Office of the Director of National Intelligence, and CISA, the agencies announced Wednesday night.

The government invoked Presidential Policy Directive-41, which outlines steps for federal cyber incident response, and Bryson Bort, founder of the SCYTHE attack emulation platform company, said he is interested to see what resources DoD provides to the rest of the government.

The agency has better resources and talent to respond, Bort pointed out, saying he will be watching the extent to which those threat teams work outside their traditional roles and collaborate with the civilian agencies.

Former IT officials pointed to Cyber Command to lead the DoD response, with each service’s dedicated cyber command working on its own network. Meanwhile in the DoD, it’s likely the deputy CIO for cybersecurity will coordinate with Cyber Command and its subordinates in the services, while also keeping the highest levels of leadership at the Pentagon informed of developments, according to Blake Moore, who served as chief of staff to the DoD CIO until this summer.

Personnel have to get a sense for what the potential compromise could entail. Just because a system was breached, that doesn’t necessarily mean the hacker affected that system, Bort said.

Even if the government disconnects from the SolarWinds software called Orion that hackers used to enter the systems, teams must figure out how far the attackers burrowed into the network. They may have transitioned to create other back doors as a means of staying in the network beyond just the original intrusion, Bort said.

To establish if they are still in the network and executing a persistent campaign requires threat hunting on the network, he added, noting this is a skillset beyond traditional IT workers. Threat hunting is something a cyber protection team would be necessary for, he said.

Each service provides these defensive teams to Cyber Command, and they act as cyber SWAT teams that respond to breaches on local networks. While the services don’t own the offensive teams they provide to Cyber Command, each service retains a select few cyber protection teams to use them how they choose within their respective service.

Their hallmark is threat hunting on a network, which involves proactively searching on the network for threat actors and requires intimate knowledge of certain actor characteristics and indicators of compromise.

In a statement to C4ISRNET earlier in the week, Cyber Command didn’t directly address whether national cyber protection teams, controlled by the Cyber National Mission Force, or DoDIN teams, which are controlled by Joint Force Headquarters-DoDIN, have been deployed as part of the response effort, but simply noted the command is postured for “swift action should any defense networks be compromised.”

It is still unclear if each service has deployed its cyber protection team in response, as spokespeople either did not respond when asked or referred questions elsewhere.

Understanding the process

When a breach like this occurs, former national security IT officials told C4ISRNET an agency first needs to inventory its systems to see if, and where, it’s running the vulnerable platform. But that’s not an easy task, said Chris Kubic, former chief information security officer of the NSA, because some agencies may not have the automated tools to identify where on their networks the compromised software resides.

“My experience is they’ve got lots of different networks and don’t necessarily have an integrated set of tools across all of those networks,” said Kubic, now the CISO for Fidelis Cybersecurity. “They probably have lots and lots of tools, but they may not be integrated together. So they may be getting what kind of little bits and pieces of the answer to that question from different tools across their infrastructure, and they’re having to try and piece it all together.”

Barrett said that officials would also evaluate the operational or business impacts of disconnecting systems that ran the software, while also mapping out what other systems a potentially compromised system was connected to.

Rick Pina, former chief technology officer of the Army, told C4ISRNET that officials would look for indicators of stolen data, newly created accounts with elevated privileges, or compromised accounts.

Officials are looking for “anything that we can capture that actually would provide a synopsis to senior leadership on … what happened,” said Pina, now chief technical advisor for World Wide Technology.

Following a 2008 breach involving USB drives, Pina said that there were daily briefings to the defense secretary, the staff and service secretaries on the aftermath and steps to deal with it.

The Defense Department’s CIO office would also have to notify Congress of the breach due to legal requirements, Moore said. The deputy CIO for cybersecurity would also review cybersecurity policies, checklists and processes to see if anything needed to be modified based on the specific event.

“If it’s something outside of CIO authorities, they’ll recommend the path to the deputy secretary of defense on how to fix it,” Moore said, now vice president of strategy and operations for Wickr, a secure collaboration platform.

This type of sophisticated intrusion poses other challenges for agencies beyond just disconnecting the compromised system from its network. Nation-state adversaries are skillful at hiding.

“Just because you remove those tools and block that access, doesn’t mean you’ve necessarily eliminated the attacker or the adversary from your environment,” Kubic said.

Assuming the adversary is hiding elsewhere in the networks, security professionals in the department or services will likely have to use sophisticated technology to uncover them, said Phil Quade, former special assistant for cyber to the NSA director and chief of the NSA Cyber Task Force.

“Longer-term, I suspect they’ll use more advanced techniques that rely on high-speed security products that allow you to do ‘break and inspect’, which enables you to look for evidence of covert command and control from operators/servers,” said Quade, now the CISO at Fortinet.

“They need to also look for evidence of the tools’ existence, even if they don’t know where they are or even if they exist, by looking for the tools’ communications with the attacker’s command and control server,” he added.