My February blog on the evolution of NSS Labs’ advanced endpoint protection (AEP) group testing discussed how the 2017 AEP Group Test produced defendable data for enterprises to use during product selection and examined what enterprises look for during product selection.
AEP vendors have captured the attention of the enterprise with some pretty bold claims about their products. Claims about detection, threat visibility, update frequency, and resource use have forced IT security teams to expand the scope of their proof of concept (PoC) testing as they attempt to measure these claims.
Results from NSS Labs’ 2018 AEP Group Test (v2.0 Test Methodology) will be released this week. Here at NSS, we are excited about the insight these results will provide for enterprises in the midst of down-selecting products prior to PoC. The test data will save time for those enterprises that do not have the resources to fully vet products, and it will provide additional information for those enterprises that do.
If your enterprise has not yet reached the PoC stage, make sure you fully understand the motivation for change. Additionally, since management console workflow and threat visibility functionality vary widely among products, make sure you allot enough time to evaluate these capabilities carefully.
Start building success criteria on day one
The first step in your product selection process should be to gather details on your environment’s requirements. This information will define your project’s success criteria, and it will help you build an accurate and defendable RFI/RFP that clearly differentiates between products.
Consider the following during your evaluation of AEP products:
- Environment requirements
- Forensic information requirements
- Threat hunting requirements
- Single-agent/single-management console or multi-agent/multi-management console considerations
- Management console feature, workflow requirements, deployment requirements
- Ancillary security capability requirements (HIPS, FIM, device control, DLP, etc.)
- Budget considerations
Security effectiveness metrics typically become more relevant during the later stages of the evaluation process (as you move closer to a PoC). During the early stages of product selection, it is safe to assume that the product you will select will provide protection that most closely aligns with your tolerance for risk. However, as with many things, there is a balance; if you start down-selecting AEP products based on security effectiveness too early, you may find yourself forced to make compromises on other features.
It’s not too early to think about management console workflow and threat visibility requirements
The products in the recent NSS Labs AEP Group Test displayed a broad range of management console workflow styles and capabilities. All meet the operational definition of AEP but have widely differing functionality and threat visibility. While management console workflow maturity does not necessarily correlate with a product’s security efficacy, it can impact policy creation and deployment significantly and should be considered carefully.
Threat visibility is a large differentiator among AEP products. While all AEP products should provide forensics, capabilities vary greatly, and not all offer the extensive forensics required by some incident response (IR) teams, such as source and callback IPs, file information, file hash, threat severity, and threat lateral motion. Additionally, attack response efforts by some IR teams require even more specialized functionality such as threat investigation workflows, endpoint queries, and more. You may not require all of this functionality, but if you do, you’ll have a limited number of products to choose from (or you’ll need a separate agent). For this reason, we recommend you determine your need for forensic information as early as possible.
AEP products have traditional drivers for change
The endpoint security market is in the midst of a renaissance that is being driven by the seemingly endless development of advanced threats and the enterprise’s need to detect them.
An enterprise’s decision to switch endpoint products is based on its need to reduce threat-associated costs and lower organizational risk. Common drivers for change include low detection rates (usually seen as an increase in trouble tickets and in time to remediate or discover a breach) or the need to improve protection against a specific threat category such as ransomware. The motivation for change may also result from a lack of satisfaction with the vendor support currently provided or the need to meet a new use case, including newer operating systems or devices. Requirements for incident resolution differ from those for basic protection and alert investigation, and they can also be motivators for change.
Finding the product that works best for your environment involves prioritizing a myriad of requirements; building a targeted RFI/RFP; evaluating product efficacy capabilities; and evaluating the product in a PoC. Is the process easy? No, but enterprises that take the time to evaluate their strategic needs are more likely to succeed in the long term.
Jason Pappalexis is managing director of the NSS Labs Enterprise Architecture Research Group. He has worked with endpoint protection products for more than 18 years and has held roles in the IT security industry that include administration, architecture, field engineering, and product testing.
The results of the 2018 NSS Labs AEP Group Test will be released at the 2018 RSA Conference. The 2018 AEP Product Selection Series will be released in Q2 and also in Q3. This series will provide strategic, tactical, and operational guidance for AEP product purchasing decisions through analysis of the capabilities within current AEP products. You can view all AEP related reports and research documents in the NSS Labs Research Library.