Share with Your Network
Late last year I was given the task of developing a simple tool that would help quantify a customer’s return on investment (ROI) in Kenna solutions. Now, first let me clarify for you that we already had an ROI tool in-house; it was really good, albeit very complicated. This new request hinged on the “simple” part of the challenge statement, and that is also what made this new request difficult.
For those who don’t want to read the behind the scenes, you can use our new tool to calculate your potential risk efficiency gains. But I hope you’ll stay with me for insights into how I developed this tool and why a remediation strategy based on a predictive model is more efficient and effective than a remediation strategy based on static scoring.
Showcasing risk efficiency
Kenna adds value and provides ROI in many ways: reporting efficiency, communications efficiency (for instance, bridging the gap between Security and IT teams), operational efficiency, etc. But the one thing that really stands out when working with Kenna is how efficient and accurate Kenna is at calculating vulnerability risk—what we sometimes call vuln risk efficiency, or just risk efficiency. So I decided that risk efficiency should be the primary focus area for an ROI calculator.
Compared to CVSS strategies
Then I needed to figure out how I could really hammer home for our audience the risk efficiency gained through Kenna. What better way than a comparative ROI tool? Ours compares Kenna’s remediation strategy, which is built into Kenna.VM, against other remediation strategies—notably the different CVSS+ remediation strategies—because they are so well understood in the VM industry. This comparison would essentially show the user that they don’t have to expend as many resources when using Kenna to get at an equivalent level of risk.
Less work for equivalent risk. Sounds good so far doesn’t it? Stay with me, it gets even better.
At Kenna, we have many ways to compare a Kenna remediation strategy against a CVSS-based remediation strategy. The easiest one to understand, and the underpinning of the simple ROI tool I ended up creating, is best illustrated using the following chart.
This chart shows Kenna’s risk efficiency compared to CVSS by comparing the distribution of all NIST NVD CVEs from 1999 onward using Kenna risk scores on the left, and CVSS scores on the right. You can think of it as report cards for CVSS and for Kenna’s risk score. As a quick aside for those of you not familiar with the Kenna Risk Score, it is based on a scale of 0 to 100, with 0 meaning your vulnerability presents the lowest risk and 100 the highest risk. Understanding that the Kenna top fix recommendations are always based on the customer’s particular environment, in general, we would strongly recommend fixing only those with a Kenna Risk Score of 67 and above, and leaving the others for another day (or until something changes and their scores elevate).
With that context established, looking at this figure, it’s clear how much better Kenna is at calculating vulnerability risk than CVSS. For me, looking at these results was one of those light bulb epiphany moments where you have to ask yourself, is this data really correct? And sure enough, the answer is yes, that data is correct. The number of CVEs in the NIST NVD database that have a Kenna vulnerability risk score of over 60 is really, truly that small!
Converting the data
Now, for the ROI tool, I just needed someone to do the math to convert the data in this chart into numbers. So I asked one of our data scientists to run the numbers for me. The data from the above chart can easily be calculated into the form of a single percentage efficiency number for each CVSS strategy compared to Kenna, i.e., CVSS 9+, CVSS 8+, CVSS 7+, etc.
95-99% efficiency
Here’s the moment where I had an even greater epiphany. I knew the data was going to be fantastic (because I know how amazing our methodology is at evaluating risk and, oh yes, because I had seen the chart). But the data (in numerical form) was so good that at first I almost didn’t believe the numbers. So I talked to one of our data scientists and verified thats the numbers are indeed correct! I suggested earlier that you give our risk efficiency calculator a try yourself. But as a sneak peek, and as a hint into why I had to double-check the data—for example Kenna is 97% more risk efficient than a CVSS 7+ remediation strategy. Yes, Kenna is that much more efficient than the CVSS-based remediation strategies.
From there the rest was fairly easy; it was just a matter of consensus building and selling the tool in-house. Building and getting internal buy-in for ROI tools is usually hard, but in this case the tool sold itself.
Check your risk efficiency today
This would be the end of the story, but marketing came knocking at my door. They liked the ROI tool and wanted something even simpler that we could offer to anyone who visits our website. And thus, what we call the risk efficiency calculator was born. Try it yourself today to see the savings over CVSS that I have noted above, but if you want to experience the full tool, with the additional information on estimated time savings, and monetary/resource savings, you’ll need to talk to a Kenna expert.
So what are the takeaways?: First, whatever you do—never settle for CVSS-based remediation strategies. As you can see, they are extremely inefficient. And second, if you do want to up your VM game, look for a strategy that uses threat intelligence and is heavily built on a rigorous data-science approach.
At Kenna, we have been fine-tuning our predictive modeling approach—and as our customers will tell you, it is powerful stuff. By combining data science with advances like threat intel analytics, we’re able to understand what’s happening now and predict with confidence what’s likely to pose a risk to your organization in the future.
And when it comes to efficiency, the numbers speak for themselves.
