Patience and persistence trump technology in SolarWinds attack

In all the furor of the SolarWinds hack, only one thing is clear: There is no defense against human error, but vigilance can mitigate disaster. In the days following the revelation of the hack, we contacted a dozen security experts. Unanimously, they saw it as meticulous work of an accomplished team, but it did not take a genius hacker team to break into SolarWinds’ development environment. It just needed patience.

Thousands of companies and government agencies use the SolarWinds Orion Platform. All it took was a simple, but yet undefined human error to allow the hackers to plant malware into a March 2020 update. From that point, the hackers were free to play hide and seek on a massive scale practically everywhere. 

We contacted several SolarWinds competitors who all claimed to have superior defenses and containment for such attacks, Our experts, however, insisted there is no defense against a single human providing an open target and none of the competitors were willing to sit for interviews to refute their findings. Moreover, while companies invest in products that claim to be absolute defenses against cybercriminals, the experts said they tend to encourage complacency.

Technology breeds complacency

Mike Stamas, a founder of GreyCastle Security, said that the industry, as a whole, did not respond well to this incident. “There was no shortage of companies saying, ‘If you used XYZ technology you would have stopped this.’ There is no better way to get yourself into a precarious position than believing that is true.”

Matthew Rosenquist, CISO for Eclipz.io, elaborated. “When we look at cybersecurity, we have to look at the technology, we have to look at the people and we have to look at the process that connects them. If you’ve got a bad user doing bad things, or less than intelligent things, it can undermine the system security, even though the system may be operating exactly as you would want it to.”

That seems to be the case in SolarWinds and potentially within the entire industry of IT management software. The company’s senior management was warned of lapses in security operations as far back as 2017, according to a December 21 report in Bloomberg News, and chose to believe they were secure anyway.

John Flory III, CISO for Harbor Networks, pointed out that SolarWinds is not in the business of cybersecurity and to expect them to be is foolish. At the same time, he pointed out, SolarWinds has access to absolutely every bit of data telemetry logs in customer environments, and unless a customer is taking the time to monitor the monitor they are unnecessarily vulnerable. 

Monitoring vendors

“I think we have to take a more micro look at response and recovery and understand where we are most vulnerable,” Flory said. ‘“I think most organizations don’t know that. They might have an idea, but they haven’t gone through the process of determining how vulnerable are the vendors? And what are they accessing? And how do I rate them?”

All of our experts agree that FireEye, even though they took much of the initial blame for the infiltration of sensitive government networks, had an incident response plan that was responsible for identifying and eventually containing the hack. Their zero-trust routine, which is common for companies where security is the business, may have prevented further harm.

That, however, is not to say that SolarWinds or any company in their industry does not have security procedures and technology in place. Perusing the SolarWinds website before they shut down access to certain pages, we were able to find a list of security protocols that mirrored that of half a dozen competitors including VMWare, Microsoft, and Dynatrace. So what happened?

Before the hack was announced, we talked with Steve Hanna, co-chair of the Embedded Systems Work Group of the Trusted Computing Group, a decades-old standards group dedicated to secure computing and his view of the problem then was simple: when people don’t use the technology or protocols established for secure computing, stuff goes bad. (The interview with Hanna and Rosenquist can be heard on Crucial Tech)

“A lot of the technology we need to secure our systems is something that we’ve already bought and paid for. As an IT professional, have you turned it on for your users? Have you encrypted their hard drives, or turned on multi-factor authentication. And have you turned it on for all those folks who are working from home?” He asked. “Probably not”

Security is worthless unless implemented

Hanna pointed out that everyone makes some investment in securing their property. A homeowner installs deadbolts on exterior doors. A business owner may buy a pulldown security screen in front of a glass door along with the deadbolt. A military base will put armed guards on the base perimeter. But when someone fails to lock the door, pull down the screen or a guard falls asleep, there is no security, especially from a criminal waiting for a human lapse.

In the SolarWinds case, that is exactly what happened. While the details are not completely clear, an employee violated protocols with an easily deciphered passcode. It is not yet known if this violation was intentional but it is what the hackers had been waiting for. The experts we talked to independently agreed the hacker team was scanning SolarWinds for a vulnerability for months, if not years, waiting for one person to provide an opening to install their sophisticated malware trojan that included a method of bypassing multi-factor authentication. However, they also agreed that SolarWinds is not the only target, just the one we know about now. Every vendor to critical infrastructure companies and agencies is also targeted.

“Organizations must have unbiased vendor management, able to rate and understand if the vendors are doing what they’re contracted to do,” said Flory. Without that the internal staff responsible for security have an incentive to cover up breaches because they were the ones who chose the vendor. 

“I talk to organizations every day that have invested in lots of cool products, but they don’t know what their top risks are,” said Stamas. “They don’t know what types of information they’re sharing with their vendors. They don’t have an effective, proven plan to execute when things go sideways. You need to have an effective team that includes people from both IT and other areas of the business including crisis communications, legal coordination, and customer relations.”