note: these are speaking notes that guided me through the speech, not a transcription. They’re also full of typos and grammar errors.

Third party Risk

Today I wanted to share some thoughts on third parties, the benefits they present for us and the challenges in assessing the risk. I believe that managing third party risk must be a core competency for security teams because our businesses depend more and more on third parties. In my opinion, there is no clearer alignment for security with business value than third party risk management. Given the importance of third party risk management, it is essential that we do it well, and to do third party risk management well, and at scale, we need to move beyond the questionnaire and find more holistic ways to engage with vendors that are a fundamental part of our digital supply chain.

So, here’s a question for all of us: are we making it easy for our vendors to do the right thing? Are we as an industry running processes that make it hard for vendors? Are we unintentionally encouraging vendors to engage in behaviours that degrade overall responses? Do we engage in partnerships or do we engage in adversarial relationships?

I think it’s important we understand why third parties are critical to our businesses. The short answer is that most businesses are focused on their core competencies; Canada Life works to protect the health and economic future of 12 million Canadians. What we don’t want to be is the best Microsoft Exchange operator so we outsource that to Microsoft. We also don’t develop software, we buy software or we integrate and configure off the shelf products. In every aspect of our business, we benefit from outside expertise and innovation so that we can focus on our core competency of servicing our customers.

Consider the dozens or hundreds or thousands of business processes in your organization; how many today are powered by a dedicated on-premises application? How many today are powered by an excel spreadsheet with complex macros shared by several users? How many are manual email workflows? Eventually a third-party solution will come along for each of those processes and the business will want to buy it. Expect the need for vendor assessments to only go up over the coming years.

The ability to assess and manage vendor risk is a core competency for security teams and its closely aligned to the needs of the business.

Today I want to talk to you about five key problems when it comes to managing third party risk.

• Goal Alignment
• Questionnaires
• Scale
• Currency
• Nested Dependencies

I’d like to share a story with you. Before I became the ciso of Canada Life I worked at a vendor. This vendor was a software as a service company who sold customer intelligence software. My team and I had two functions, Make our product and company as secure as possible AND convince our customers our product was sufficiently secure.

One of my favourite memories was a discussion between a company lawyer and I (we had three lawyers and five security people which was pretty good for a company with less than 1000 employees). So there I am in the lawyer’s office and we’re discussing a customer contract issue. My legal colleague says to me “that’s a complex issue“ to which I respond “it’s really a simple issue, if we are a company that follows the law and if we are a company that tells our customers the truth then this is pretty straightforward.” As those words come out of my mouth, the chief marketing officer sticks his head into the office and says “those are words I never want to hear being discussed by our head of security with legal” and promptly walk off

When I took on the role at the vendor I was new to selling security as part of product. Security by itself, no problem, Security as part of the value proposition of selling the product was New Territory for me. In the first few weeks in my new role I made the almost fatal mistake of sharing too much information with a customer about a recent penetration testing report. I nearly knocked the deal off the rails. I learned quickly that’s the purpose of security in a sales setting was to the help get a deal through.

I was not goal aligned with my organization

Security people inside a vendor org are there to help the sale get through and to avoid the loss of future customers. That is their goal alignment whether they know it or not. Security people at vendor orgs that don’t line up on that may have shorter careers. The idea that vendors should want to do security because their customers demand it only goes so far in practice. You work with your business colleagues… you know how much security they really want.

A vendor will only do security as far as it removes a disincentive to buy their product… and remember who the primary buyer is… not you, not the security org, it’s usually business people who buy on value, not security. Vendors know this, they know they need just enough security for the average deal.

The salesperson’s second job, after qualifying the lead, is to remove enough friction so the value proposition becomes strong enough so that the deal goes through. At my time vendor side, the number one question from sales without fail was “how do I bypass the security questionnaire?”

Which brings us to the questionnaire.

For a vendor the questionnaire is an obstacle to be circumvented and worked through as quickly as possible if needed. The security teams inside a vendor grow to despise these questionnaires, one a month, a week, one a day. Always asking the same questions, but always in slightly different ways. However none of them ever getting at the core of it. I had a saying within my team… “we never lie to the customer, but if they don’t know how to ask the right questions, that’s on them”. Over hundreds of questionnaires and follow-ups, not a single customer security team ever asked about the support access backdoor that could bypass all security on the platform. A questionnaire will never give us the whole picture and even if it could, the people answering them are incentivized to say as little as possible.

In my prior role we came up with smart ideas to address the workload of the never-ending stream of questionnaires. We licensed Shared Assessment’s SIG and completed that and we offered it to our customers instead of their form. A few agreed, but most wanted their questionnaire filled in anyways. We built a library of pre-canned responses and tried to teach sales engineers how to answer the forms; that worked a little but had limited utility and took the sales engineers away from actual selling. We decided to invest in a SOC2 and a quarter million dollars later learned that our customers were very happy that we had a SOC2 but still wanted us to fill in the form anyways. With the unrelenting pressure to sell, combined with the unending flow of security questionnaires our answers by necessity became short and simple.

The results of many customers trying to do the right thing to protect their data had the unintended consequence of degrading the overall quality of responses as we got shorter and shorter with our written replies. The irony of it was it took the security people away from actually improving things… our improvements were better selling outcomes… not better security.

During my years at that vendor, we lost only lost three deals for security related reasons: two because of regulatory issues we couldn’t address and one because they wanted a type of security technology that didn’t make sense for us to deploy. That leads me to believe either our business value was so compelling that it overrode any concerns, or that our responses were just putting a check mark in a compliance box; I’ll leave you to draw your own conclusions.

Even better mousetraps like shared assessment platforms are just more efficient versions of the same problem. Even if you could make it painless for a vendor to respond to questionnaires… the answers would still be incomplete because of who is answering them and why they are answering them. There is an asymmetry of information and context - the vendor will always know their product, their platform, their code better than you, and are not goal aligned with you.

As financial institutions we can demand SOC2 assessments, ISO 27001 certifications or anything else… but remember who pays for the certifications…. The vendor. And the people that perform the certifications want the vendor to pass because they want repeat business. I’m not saying there isn’t diligence and integrity on the part of the auditors… rather, there is coaching and shaping of the responses. I know this is cynical, but it’s also real - always think about who pays.

The next challenge for vendors is their scale. If you’re dealing with Microsoft, Amazon or Google then their security team is bigger and better than yours. But that is not the majority of companies. Most technology vendors are much smaller with a handful of security people. Generally we want to do business with these smaller orgs

Consider for a moment the number of tasks required to run a fully functioning security operation - the processes, tools and governance is generally all the same. If you’re a systemically important bank you do all those processes across thousands of pieces of technology and you deliver it across many lines of business. As you shrink down in organizational size, those tasks remain, you’re just performing them over fewer lines of business and fewer systems. As you shrink, you need fewer people and eventually you get to a point where one person might be able to deliver several security functions in an organization small enough. But there is some lower limit. You cannot run an entire security program with a single person… or perhaps with a few single digits of people.

The budget scales down in the same way. Sure, a scanning license costs less when you have fewer IP addresses, but costs don’t scale down in a linear fashion and there is some fixed cost that you can’t get below. Small companies need to spend more proportionately to achieve some reasonable level of security. Every dollar spent needs to result in a return; and unless buying security tools guarantees more sales, then it’s not worth it to buy that tool just one customer aske for.

As we engage with vendors we need to recognize that they operate at a different scale to us. That even if they say yes to everything in the security questionnaire, they’re probably performing the majority of security functions superficially. That doesn’t result in real security.

Our fourth challenge is simple… it’s about change. Smaller companies change faster than large companies. The half-life of security questionnaire or any security assessment is likely measured in months. All it takes is one new feature to radically change the security properties of a system.

We’ve implicitly accepted this idea of drift, with vendor updates much less frequent than the rate of significant change.

Our final big challenge is broader than third party risk alone, it is the digital supply chain which is made up of third parties, fourth parties and off the shelf technology providers. It’s also made up of businesses that appear to be pure service organizations but have a lot of technology under the covers.

In the last few months, the challenges of securing the digital supply chain have been highlighted to us by the the solarwinds hack from December 2020. I suspect several of us in the audience had a challenging few days as we assessed the risk and took appropriate action. With perfect hindsight I ask how many of us assessed SolarWinds as something more than a commercial product? Perhaps we asked some questions about their software development practices? How many of us asked about the security of their software update process? Even if you did ask I don’t think you would have received anything other than a glowing response as evidenced by SolarWinds breathy blog posts in the prior year about their secure software development. I suspect many security professionals would have treated a piece of on premise software as nothing more than just that; a piece of software that was likely low risk for some given it didn’t process data… yet in hindsight it had access to an awful lot of important data.

I think to many vendor assessment processes, the idea that the build pipeline could be compromised wasn’t contemplated. That software could be compromised is not a new idea; Russia compromised a significant Ukrainian tax software vendor to spread NOTPETYA in 2017. Every few years some threat actor compromises a well deployed piece of software as a means to access their trusted user base; it’s a useful tactic and one that takes advantage not only of the unsuspecting user base who trusts the vendor but possibly even takes advantage of the good security practices to keep your software updated.

There are many possible responses on how to buffer against these sorts of attacks and I could speak for hours, as could many of you, on the different approaches to secure the software development processes. One of those processes is known as reproducible builds. It’s not hard to implement but what if the vendor had never heard of the ideas? Could we educate them so that they build that assurance over the software pipeline? What if the vendor didn’t see this as important? Perhaps they see themselves as just a software vendor; they sell you access to a blob of compiled bits; to them it’s quite transactional but to us they’re a key part of our digital supply chain. If they are compromised, we are compromised.

There’s also the problem of dependencies inside software code. Whether our own internally developed software, software powered cloud platforms or off the shelf software they all have one thing in common. Software is built up from modules that provide various utilities; developers add the unique business logic and glue processes to connect the code together.

As a hobbyist developer, my python code usually has several software modules imported. The same is true for my Javascript code and when I refer to the modules I import I’m referring to the top most modules that I want to consume but those modules often have dozen or hundreds of underlying dependencies. From a security perspective, it’s probably turtles all the way down when it comes trying to secure the code from malicious modification. All it takes is one little library deep in the dependency tree to cause damage.

In 1983, Ken Thompson delivered an important talk. Ken Thompson designed and implemented the original Unix at Bell Labs. His seminal talk – “Reflections on Trusting Trust”, was delivered as a Turing Award Lecture. His idea, back then, was that software code, libraries and compilers could all be backdoored – he coined that idea. While the technical elegance of his talk is to be admired, what’s more important is the recognition that developers will trust code and compilers from sources that they can’t verify. Back then, public repositories of code modules didn’t exist. Today there are millions of third-party libraries across dozens of popular languages. These repositories are not app stores, the modules are not curated. Anyone can upload a module; in some cases modules are abandoned by their maintainers and taken over by other maintainers. Sometimes those new maintainers have malicious intent; for example a javascript module in 2018 was modified to steal bitcoin private keys when the module was included in a specific payment platform.

So this is the last big challenge I ask that you think about. The digital supply chain is not only outside our organizations, it is inside our own organizations. Sometimes a third party is one you didn’t know you had. We have development processes that are dedicated to bringing in third party code into our environments; third party code that has been backdoored. That same problem exists for our vendors. I think this problem is going to be hard to solve because we will depend on not only secure develop practices within our own organization but also secure development practices for every developer of every module we consume. We will not only depend on secure development practices but also assurance around the development practices themselves, again both within our organization and for every outside module maintainer we depend on.

So if we think about these problem spaces: goal alignment, knowledge asymmetry, scale, currency and the web of dependencies I think it becomes clear that questionnaires, while an important tool, is a limited tool. It may be one that gives us a false sense of comfort.

We may choose to accept the risk associated with the limitation of these security tools. But I don’t think we’ve made that explicit decision as a profession… if anything we’ve committed to making the questionnaire more efficient with shared assessment models such as Hi-trust.

I think we need a different way. I think that different was means changing the nature of the relationship with the vendors. I don’t pretend to have a solution but rather a framing, a north star we might head towards.

If we agree that the web of vendors are actually key to our success, then can we frame them as partners and not adversaries.

If they are our partners, then we want to invest in them because if they are more successful then we are more successful.

Partnership means ongoing engagement.

I propose that the start of the vendor assessment is not the questionnaire but rather learning the vendor. Learn their business, learn their technology architecture and how they manage it. Not through dry questionnaires alone but by actually talking to them. It’s an investment, but you will learn more this way. Sure, still do all the questionnaires and SOC2’s you think are necessary, but build an actual relationship. You will learn far more through sitting down with people.

Then don’t demand they fix everything, but rather spend time understanding their risk prioritization process. Coach them if needed. Make sure that they have a robust risk management process embedded in their organization by observing it in action. Do this rather than demanding that a specific set of vulnerabilities be fixed.

Work to understand the scale that they operate and what capabilities are achievable for them on a risk prioritized basis. Find ways to extend your vulnerability intelligence and threat intelligence programs to your vendors. Share the best of your security program; give these vendors access to the thinking and patterns you have developed. Consider education and awareness for your vendor community. Our education of vendors not only needs to be about what to do, but also why to do it. To actively manage our digital supply chain we need to convince our vendors that they are a part of it; that their obligations to us go well beyond the transactional delivery of compiled software or the maintenance agreements. We need them to believe that because anything less means we will get a partially answered questionnaires and surface level attestation reports.

Finally engage often; the rate of change inside smaller companies is much faster than in financial institutions. Don’t send them a form once a year. Reach out quarterly to understand how their technology has evolved and what new risks that brings.

That, I think is an opportunity for the security profession, for this community… whether through consortiums or industry groups, to educate and nurture smaller organizations on their security journey. Outreach and partnership may be one of the ways we produce a scalable process for actively managing security risk.

With that I thank you for listening; I hope you enjoy the rest of the conference.