Several years ago a Cisco announced a critical vulnerability in their firewall platform. It allowed an attacker to gain remote access to the firewall and the network beyond. I directed those affected that I worked with to shutdown their firewalls which meant no internet and no site-to-site VPNs. Business was impacted for several days.

A few years ago I came across a report that detailed the exploitation and use of that same vulnerability by a sophisticated threat actor. I felt vindicated! I had made the right call. I had done the right thing at the time… but… had I? Or was I lucky?

Incomplete information forces us to make decisions that have uncertain outcomes… we may actually make these decisions every day… and you’re comfortable with it in many aspects of your life. You drive a car, you don’t know exactly what will happen and while we have a reasonably good intuition about the outcomes, there’s a non-zero chance you will be in a car accident.

When the SolarWinds breach was announced I imagine many companies discussed should they remove it from their network or not on that challenging December 2020 night. CISA directed agencies without necessary skill sets to power down their Orion deployments.

Shutting down Orion was a safe action but at the time nobody, except perhaps a select group of employees at FireEye, SolarWinds and intelligence agencies knew who had been impacted; which networks Russian Intelligence had actually compromised. I speculate that for some organizations, shutting down their Orion product for a short while was survivable, but that all depended on the size and complexity of the network as well as the purpose Orion was used for (some functionality may have have been mission critical).

In hindsight we have a better understanding of who the SolarWinds threat actors were after, primarily but not exclusively government agencies, and the vast majority of companies running the Orion product were unaffected. For the organizations that depended on Orion to run their network and systems, they had a difficult choice without the benefit of hindsight.

Shutting down Orion and shutting down the Cisco ASA firewalls were safe actions but they weren’t necessarily the correct action. An organization could have waited till it knew more… if the threat actors were actually exploiting the vulnerabilities/backdoors.

What about the inverse? Not taking action? Just waiting? Waiting is often the low cost option and you can always take action later.

There’s often an information asymmetry. You likely understand the vulnerability well and you have some intelligence on the threat actor’s techniques and indicators of compromise. You may or may not know the threat actors target selection approach (or approaches if there are multiple threat actors exploiting the vulnerability). You definitely don’t know when a threat actor might actually attack your organization.

In the face of that information asymmetry we still have to make decisions and those decisions could prove wrong in hindsight. If we’re willing to make decisions with imperfect information then we implicitly are willing to accept the risk of being wrong. At least some of our appetite for risk comes from be willing to accept the potential for loss that comes out of making decisions without knowing everything.