How fast should you fix a security flaw? The intuition is as fast as possible. We don’t know when the vulnerability will be exploited and it may never be exploited. We set heuristic rules for how fast vulnerabilities of certain severities should be fixed; are those rules right?

The recent Log4j vulnerability and subsequent mass exploitation (Log4Shell) might at first glance suggest fix everything immediately. However, the actual timelines were longer as the vulnerability had existed in the code since 2013 and mass exploitation only started on December 9th (reference: wikipedia).

Remember HeartBleed? Public exploitation started within 24 hours of disclosure. That data point suggests to fix it now or shut it down now (the “burn it with fire option”).

Some vulnerabilities are popular and get exploited often and some might be exploited infrequently or perhaps not all.

As CISA explains for Directive 22-01:

… many vulnerabilities classified as “critical” are highly complex and have never been seen exploited in the wild - in fact, only 4% of the total number of CVEs have been publicly exploited. But threat actors are extremely fast to exploit their vulnerabilities of choice: of those 4% of known exploited CVEs, 42% are being used on day 0 of disclosure; 50% within 2 days; and 75% within 28 days! - source

Consider a vulnerability without exploit code in some deep part of Linux… it’s technically exploitable, but most likely won’t be, especially if it’s a local exploit only. Next consider a vulnerability in a proprietary application your organization built that is buried past a layer of authentication and the likelihood of your average customer finding it is extremely low. If you’re selling a specialized tax management tool to Danish cattle farmers there’s a reasonably good chance that vulnerability nevers get exploited.

We fix vulnerabilities for two reasons: first, so that we don’t break the security promise; and second to stay on the currency path in the event there is a future patch we really do want to apply. The first reason is universal, the second only applies to commercial software. The security promise is that your system is secure to the best of your knowledge; if we now know about a vulnerability then we need to fix it or the system cannot achieve it’s promise of being secure. Now I suppose one could never look for vulnerabilities in the first place or set some sort of arbitrary restriction on what types of vulnerabilities to look for, but that approach has obvious problems with it not least of which is you might miss something really important that is likely to be exploited.

We don’t know in advance which vulnerabilities will be exploited, although I suspect we can guess the most likely to be used by threat actors as they’ll probably have two factors going for them - being widely available and being internet accessible. So in the face of that uncertainty we have heuristics, which say patch everything within a certain number of days for a certain level of severity see PCI DSS 3.2.1’s Requirement 6.2. Even CVSS is a heuristic mechanism; it’s an algorithm representing the thinking of the a group of security experts.

Are those heuristics still needed given what CISA has set out in it’s directive? and what Ed Bellis and his team have advocated for years?

Palo Alto analyzed ExploitDB, a public Exploit Database, and found that of “45,450 public vulnerabilities” ~24% had exploits. Of those “11,079 exploits” 14% were zero-day which they defined as an exploit “published before the vendors release the patch” for the vulnerability. Of those 23% of the exploits were published within a week after the patch release and 50% of the exploits were published within a month after the patch release. The researchers go on to conclude that the risk of a vulnerability being exploited increases quickly after vendors release the patches. (note: I’m cautious as to the bias of embedded research teams, but Unit 42 did use open data and a research paper on “Historical Analysis of Exploit Availability Timelines” noted similar observations).

I think that’s a reasonable hypothesis on the part of Palo Alto’s research team, the idea that a vulnerability is more likely to be exploited if there’s a exploit code out there makes sense. Roughly 3.4% of vulnerabilities, which is 1,551 vulnerabilities out of a total 45,450 vulnerabilities, had exploits before they were published; and 87% of exploits were released within a month of the vulnerability being disclosed (which is ~21% of published vulnerabilities). HD Moore, creator of Metasploit, gave rise to HDMoore’s Law which is “Casual Attacker power grows at the rate of Metasploit”; said differently, if you make it easier to exploit a vulnerability it’s going to get exploited. Give everyone easy access to a weapon and it’s likely to be used…

Here we have these two ideas:

1. The vast majority (87%) of disclosed vulnerabilities have exploit code available within one month; and
2. Most vulnerabilities are not actually exploited (or perhaps not exploited that much).

However, that analysis is focused on commercial off the shelf software; none of these data points are inclusive of the vulnerabilities in an organization’s proprietary software (like your cow tax app). Common Vulnerability Enumeration (CVE) are the unique identifiers for vulnerabilities found in software; CVE Numbering Authorities are responsible for issuing these unique identifiers and their rules exclude issuance of a CVE for proprietary software an organization might create for their own business purposes (that is, not selling or distributing the software).

The analysis of CVE tracked vulnerabilities is limited to distributed and licensed software, which leaves the question of how fast is your proprietary business application likely to be compromised? I’ll focus on web applications as that seems to me to be the dominant form of application being developed these days.

I think for web apps the act of discovery is likely the act of compromise or at least happens very close to each other. With the exception of white-hat security researchers and penetration testers, if someone is searching your application for a vulnerability, they most likely mean to exploit it. There’s no CVE database of equivalent, rarely is there public disclosure except for the occasional after the fact bug bounty write-up. I was not able to locate application specific data for Mean Time To Exploit or Mean Time To Identify the original vulnerability. That’s not say it doesn’t exist, I just have finite time to search.

Contrast Labs' Application Security Observability Report shares some statistics on application specific remediation timelines. Sure, it’s a vendor report but given Contrast Lab’s core business of security web applications it’s hopefully based on solid, but proprietary, data. Of note, they share remediation timelines:

… Contrast customers achieved a median time to remediate of seven days as compared to 121 days for customers of one static application security testing (SAST) vendor. The differences are even more dramatic when serious vulnerabilities are examined, with 25% being remediated in one day and 75% in 16 days. - source

Unfortunately we don’t know what Mean Time To Identify was, those vulnerabilities could have been in the code for days, weeks or more. However, from a risk management perspective, one does not want to worse than the average so at least that is something to aim for; don’t be worse than the 75% of organizations that fixed a serious vulnerability in 16 days (per the Contrast Lab’s report). If we swing back to Log4j/Log4Shell, that vulnerability existed for eight years before being found and exploited, so perhaps there is some comfort that vulnerabilities can remain undetected for some time. However, if your web application is built on a common framework (think Ruby on Rails, React, Django, Spring, Struts etc…) which I generally think is a good thing as a means to not reinvent wheels or make obvious mistakes, then you do inherit some aspects of vulnerability identification followed by mass exploitation.

At the start of this post, I said “we set heuristic rules for how fast vulnerabilities of certain severities should be fixed; are those rules right?". I think that for commercial/distributed software vulnerabilities, we likely have a lot of data about how fast we need to fix based on how rapidly the vulnerabilities tend to be exploited. For vulnerabilities in proprietary software, such as an organization’s web application, we likely do not have enough data. So for now, perhaps it’s just simplest to continue to rely on a heuristic that says: if this severity then fix it within that many days. In a complex technical environment with information gaps and information asymmetry along with multiple data points to consider on a per vulnerability case and then likely having to engage with many stakeholders, keeping it simple is probably the most effective and risk appropriate approach.