The most useful metric

Sun, Nov 21, 2021 2-minute read

Over my career I’ve designed a lot of metrics; some colleagues have occasionally suggested I’ve designed too many. Over the past few years I’ve started thinking that less is more, so if I had to choose one single metric to assess the security performance of an organization, it’s Mean Time To Remediate (MTTR).

MTTR is, in my opinion, a great measure for several reasons:

  1. It can apply to lots of situations (vulnerabilities, compliance issues, audit gaps, third party issues) and it easy to understand for technical and non-technical audiences;
  2. It shows how fast the organization can fix a problem which is an implicit indicator of priorities;
  3. It’s not sensitive to a mass of new problems coming into the environment (which is not unusual on a specific Tuesday each month);
  4. It’s a forward looking indicator. Past performance is a good indicator of future performance (unless there’s a sea change);
  5. It’s easy to align with policy and risk appetites; and
  6. For some use cases it’s easy to measure with some basic tooling and math.

MTTR is easiest (and I believe most appicable) to infrastructure vulnerabilities, the sort you can find with a vulnerability scanner that enumerates CVEs but it’s applications are so much broader.

Deciding on appropriate target for MTTR is also relatively straightforward; just line it up with your regulation or industry data. For example, PCI-DSS specifies (in section 6.2) that “security patches for critical or at-risk systems are installed within 30 days”. Alternately, research suggests most vulnerabilities have published exploits before patches are available so you want to patch as soon as you can and mitigate even faster; so a lower threashold may be appropriate. Deciding on thresholds where you don’t have regulation of industry data to inform can take a policy driven approach or an iterative approach of starting somewhere and shifting the target as one learns about the consequences of higher or lower values.

*Here’s two question to ponder on: When should we start the MTTR timer? What is the impact of starting it earlier? Some thoughts here in Start your MTTR timelines earlier.