Press

NSS Labs Announces 2018 Breach Detection Systems Group Test Results

October 11, 2018
Three Products Receive Recommended Rating; Two Products Miss at Least One Evasion

AUSTIN, Texas – October 11, 2018 – NSS Labs, Inc., a global leader and trusted source for independent, fact-based cybersecurity guidance, today announced the results of its 2018 Breach Detection Systems (BDS) Group Test. In this fifth iteration of the BDS Group Test, three products from market-leading security vendors were evaluated for security effectiveness, total cost of ownership (TCO), and performance.

According to an NSS Labs research survey, BDS are most commonly deployed by the banking industry (49%) and the durables industry (manufacturing) (41%). Survey participants cited false positives and a lack of corporate image support as challenges experienced with various BDS products.

A BDS is designed to detect and log both successful and attempted breaches in an accurate and timely manner, while remaining resistant to false positives. BDS utilize both static and dynamic analysis techniques to detect advanced malware, zero-day attacks, and targeted attacks that have bypassed network security controls. Through constant analysis of suspicious code and identification of communication with malicious hosts, BDS are capable of providing enhanced detection of threats. Such threats range from commodity malware to targeted attacks from state-sponsored threat actors that could bypass traditional network defenses such as next generation firewalls (NGFWs) and next generation intrusion prevention systems (NGIPS).

Key Takeaways

• The most important metric to consider for BDS is Time to Detect. The time it takes for attempted or successful breaches to be detected can influence the overall impact of an attack. The more time an adversary has to operate after breaching a network, the greater the possible damage.
• For a BDS product, rapid detection and analysis of both successful and attempted breaches is critical in halting the damage caused by potential malware infections or breaches.
• Attackers use evasions to bypass security controls. A single evasion can grant an attacker access to your network. In the 2018 BDS Group Test, products were tested against 374 evasions to evaluate how well they were able to detect the evasions.
• The resiliency of a system can be defined as its ability to absorb an attack and reorganize around a threat. NSS Labs measured the resiliency of BDS products by introducing a vulnerability along with its triggers and then asking the products to detect it. One product demonstrated full resilience against attack variants.

The 2018 NSS Labs BDS Group Test included:

  • Hundreds of victim machines
  • Collection and analysis of Terabytes of logs
  • More than 2,400 attacks, which included 374 unique evasion samples
  • Hundreds of discrete samples used by threat actors in current campaigns
  • Exploits, malware, and evasion testing using regularly abused compromised mediums such as web and email and leveraging multiple common document types.

“Breach detection systems (BDS) attempt to discover attacks that can bypass traditional security controls by examining various indicators of compromise (IoCs) in order to determine whether files are malicious,” said Jason Brvenik, Chief Technology Officer at NSS Labs. “The ability of a BDS to detect and report on successful compromise in a timely manner is critical to maintaining the security and functionality of the monitored network. We encourage enterprises looking to purchase a BDS to review the 2018 BDS Group Test findings for insights regarding which product provides the best protection and value for their organization.”

The following products were tested:

  • Fortinet FortiSandbox-2000E v. 3.0.0 & FortiClient (ATP Agent) v.5.6.6.1167
  • Lastline Enterprise (Sensor 1000) v8.0
  • Trend Micro Deep Discovery Inspector Model 4000 (Hardware model 4100) v5.0 & OfficeScan XG SP1

Unverified Products:

  • FireEye
  • Cisco

As with all NSS Labs group tests, there is no fee for participation, and the test methodology is available in the public domain to provide transparency and to help enterprises understand the factors behind test results.