Prioritize Open Source Risks with Endor Labs
This article is part two of a 3-part series on vulnerability prioritization workflows:
- Discover pen Source Risks with Endor Labs (Part 1)
- Prioritize Open Source Risks with Endor Labs (this article)
- Fix Open Source Risks with Endor Labs (Part 3)
The “vulnerability prioritization workflow” is one of the primary areas of responsibility for an AppSec team. And while that workflow will vary from place to place, they can all be summarized in a three-step process:
- Discover risks— Identify the vulnerabilities that could threaten your business.
- Prioritize risks— Eliminate irrelevant findings and determine are the greatest threats.
- Address risks— Accept risk, remediate, or mitigate the vulnerabilities.
Software composition analysis (SCA) tools are a common part of this workflow, used to automate toil so AppSec owners with limited resources can focus on higher-order tasks. In this series, we talk about the role SCA can play, shortcomings you might encounter, and what “better” looks like. We covered identifying risk in the first blog of this series and now we tackle the process of prioritizing risk.
Too many vulnerabilities, what do you prioritize?
For most organizations, fixing every single vulnerability discovered by an SCA tool isn’t feasible — and certainly not a good use of valuable AppSec and developer time. Risk isn’t a one-size-fits-all concept, meaning a risk to organization A might not apply at all to organization B. Teams typically start with an internal definition of risk tolerance (especially if they have to comply with a standard like PCI DSS) and then compare each risk to that standard. To facilitate this process effectively, the AppSec team needs to know how each risk potentially impacts the organization.
The problem with CVSS
Standards are a valuable way to create consistency in a program, and most AppSec programs use a CVSS (Common Vulnerability Scoring System) base score as a starting point to decide which risks to remediate. For example, it’s typical to prioritize findings by critical and high (deprioritizing the medium and low scores).
CVSS is a measurement of how bad it could be if a vulnerability is exploited. Filtering by severity helps, but even the CVSS itself knows it’s not really adequate unless the organization is also analyzing each vulnerability based on how the affected component is used in their environment, and correcting the CVSS score with that data. Very few organizations have the resources to do this for ever CVE, so they tend to take the score at face value. It’s also worth noting that CVSS scores are somewhat biased (researchers tend to over-score, and vendors tend to under-score), so it’s an imperfect option. Given the reality of how CVSS scores are typically used, it means that organizations are:
- Ignoring some things they probably should be addressing
- Spending time remediating some things they probably don’t need to
- (If they’re trying to improve the two above) Spending a lot of time and effort on analysis.
Function-level reachability for prioritization
If CVSS isn’t telling you Does this actually matter to me?, you might wonder how do you answer that question. The most reliable way to determine what matters is function-level reachability analysis, which tells you whether the vulnerability can be exploited at the function level. When a person or tool can reliably determine reachability and marks a vulnerability as unreachable, then there’s no risk of exploitation. Deprioritize the finding and move on!
Our research shows that 90.5% of vulnerabilities are unreachable, telling you that “no one can reach the vulnerable function in my app, so no one can exploit the vulnerability, so I don’t have meaningful risk.” Determining whether a possible vulnerability is even reachable is a powerful and cost-effective tool for reducing the scope of remediation work.
Traditional SCA problems with prioritizing risk
Most traditional SCA offer CVSS scores as a filter but many are adopting reachability features to meet the demand to do better with prioritization. But determining reachability is a very difficult technical challenge (as anyone who has done it manually can attest) so it’s important to know how the feature works before trusting it to do your prioritization.
Inaccurate Reachability
If an SCA tool struggles with constructing accurate software inventories, then its reachability quality also suffers — if you don’t know what you have, you can’t accurately assess whether you’re affected by a risk.
When one of our customers has been burned by “bad reachability,” it can usually be blamed on one of three traps:
- Trap 1: It trusts the import statements
An SCA tool that uses manifests to create a software inventory is also limited to using the manifest for reachability. If you wouldn’t accept an inaccurate inventory, then it stands to reason that inaccurate reachability filtering is also unacceptable. - Trap #2: The analysis stops at direct dependencies
While function-level reachability is becoming the standard for reachability, not all tools will provide it for your transitive dependencies — the dependencies that your developers didn’t pick directly, but came along for the ride. With more than 95% of vulnerabilities stemming from transitive dependencies, a tool that can only do reachability for direct dependencies will have limited usefulness for reducing noise and accurately identifying true risk. Unfortunately, many traditional SCA tools either ignore transitive dependencies entirely, or fail to conduct reachability analysis on vulnerabilities in transitive dependencies. - Trap #3: Fails to properly analyze fix commits for vulnerable functions
Some tools will look at fix commits — the log of code changes that include the fix for the reported vulnerability — and assume every change in that commit is related to the vulnerability.. But in reality, fix commits often document several changes to several functions (not just the one with the vulnerability in it). In this case, the tool incorrectly thinks a collection of functions are vulnerable, when it may only be one function. When tools do this, it increases the rate of false positives — that is, the tool more often claims a vulnerability is relevant to you when in fact it is not.
Clunky Prioritization Policies
You can have all the data in the world, but most SCA woes are rooted in the act of throwing thousands of vulnerabilities over the wall to engineering, without a reliable prioritization mechanism. This results in overwhelmed engineering teams that are either slowed down by having to analyze and argue about which of these really need to be fixed, or in some cases just outright ignoring all or most of the issues. In situations like that, security can’t even begin to have a discussion with engineering about “breaking builds” (automatically preventing serious, known vulnerabilities from being released into production) — the trust that such a policy requires just isn’t there.
Is SCA dead? There is a better way.
The problems with traditional SCA tools all stem from the same source: They don’t understand how your 3rd-party dependencies interact with your application code. A tool that doesn’t understand your application can’t tell you which potential risks are relevant to you.
With this reality as the status quo, it’s no wonder that it takes an average of 8 hours to investigate each vulnerability [pdf]. Goodbye, developer productivity; Hello, developer productivity tax. Not only is it costly and time consuming to research vulnerabilities, but it’s also error prone because it relies on individuals having good research skills and the time available to do a good job. The outcome is an inefficient vulnerability management that doesn’t succeed in effectively managing risk.
There is a better way.
Endor Labs was founded on the idea that we should expect more from SCA. Specifically, your tool should do more than show you problems: It should also give you solutions. To that end, we broke down all the assumptions about how an SCA tool should function and came up with something entirely new. Using program analysis, Endor Labs creates a simulation of your application at the time of build using a combination of the information from your package manager and analysis of your code. The output is an accurate software inventory based on how the code you write interacts with the code you reuse (your software dependencies).
Prioritize risks with Endor Labs
When an SCA tool understands how your application works (including direct and transitive dependencies), you can eliminate most of the noise because you know what is reachable at the function level, and therefore which risks actually affect you.
In addition to function-level reachability, Endor Labs provides several filters to help you decide which risks to address first, resulting in an average 92% noise reduction. Answer the following questions about each finding, which lets you decide what to monitor, what to add to the backlog, and what to fix right away.
- Is it in production code (not test code)?
- Is there a fix available?
- Is the affected function reachable?
- Is there a high probability of exploit (high EPSS)?
- How severe could the impact be (CVSS)?
- How difficult / complex is the dependency to upgrade? (more on this in the next article)
When a new vulnerability hits headlines, security engineers can:
- Prioritize if the vulnerability is relevant in the context of the application
- Prioritize the urgency of the fixes
Automate prioritization before code ships to production
As your program matures, how the SCA tool helps you automate prioritization can be just as important as what you can use to prioritize. Endor Labs enables you to automate prioritization workflows with a high degree of customization. For example, an automated workflow could look like this:
- Identify major risks— For example, you might want to trigger this policy when the finding is critical or high, has a fix available, is function-reachable, is in production, and has an EPSS score of 1%.
- Take appropriate action— Continuing the above example, you probably want to break the build because these parameters represent substantial risk. But in other scenarios you might want to warn or just send a notification.
Unlike traditional SCA tools with limited policy creation and mechanisms, Endor Labs allows users to create fine-grained action policies to dictate when to create a ticket, or even prevent a serious threat from being implemented into your software. The fine-grained policies aren't simply "if it's critical and fixable, then take an action" - users can choose to take an action if the vulnerability is critical, fixable, reachable, a direct or transitive dependency, with an EPSS of a certain percentage, in a certain ecosystem, in a certain project, with certain tags. Having this level of detail for action policies allows:
- A drastic decline in the number of broken builds, since you're only taking action on the most serious threats
- Mitigates the negative outlook on broken builds as "just another annoyance". The attitude of a broken build as a result of a security scan shouldn't be seen as a hindrance to development - it should be seen as a rare occasion that, if triggered, should be taken seriously and with caution.
Endor Labs also offers Finding policies, allowing you to raise risk or information about your open source posture. Several Endor Labs customers use Finding policies to identify End of Life software, unmaintained or archived repositories, open source projects with a single maintainer, open source components that are several months behind a current version, components with very low activity, and much more.
SCA that meets your expectations
With the Endor Labs approach to SCA, you can stop stressing about hidden risks, noisy tools, and mysterious upgrades. We provide:
- Faster risk reduction
- Improved developer productivity
- Accelerated compliance management
Stay tuned for the next article where we delve into the next step: Fixing Risks with Endor Labs.
Book a 20-minute demo to learn how Endor Labs turns your vulnerability prioritization workflows dreams into a reality or start a free, full featured 30-day trial that includes test projects and the ability to scan your own projects.