Blog

Breach Detection Systems – Precious Metal Projectile or Pale Pachyderm?

April 2, 2014

NSS once asked: “Will breach detection systems become the latest security “silver bullet” – or a “white elephant?” Precious Metal Projectile or Pale Pachyderm?

To answer this question, we must review the NSS definition of a breach detection system (BDS), as seen in the Breach Detection System Buyer’s Guide:

“NSS defines BDS as systems that are implemented to identify and report actual breaches as well as attempted breaches. Many BDS products on the market can identify an attempted breach by the download, or “drop,” of a file that the BDS product knows to be malware. An actual breach occurs when the downloaded file is executed, and the workstation performs the activity intended by the malware. This activity may include the workstation sending communications to a command & control (C&C) server, or even polling the Domain Name System (DNS) in an attempt to contact the C&C. An attempted breach occurs when the “drop” contains a payload that is not compatible with the workstation.”

As with any emerging market for security technology, there are many claims that must be verified independently, and this is where NSS Labs comes in.

The NSS Labs Breach Detection Systems Security Value MapTM (SVM) was released today and includes 6 leading BDS vendors: AhnLab, Fidelis, FireEye, Fortinet, Sourcefire (Cisco), and Trend Micro. To see how your preferred BDS vendor performed, download a copy of the BDS SVM graphic. For additional information on the performance of each of the vendors tested, see the individual product analysis reports (PARs).

NSS’ testing and evaluation of BDS products has been a complex and insightful process that has taken more than a year to complete. NSS formulated this new market segment in late 2012, and since then, NSS has published several research papers on the topic, including:

  • Breach Detection: Don’t Fall Prey to Targeted Attacks (November 2012)
  • BDS methodology (January 2013)
  • Breach Detection Systems Buyer’s Guide (July 2013)

In the BDS group test, NSS evaluated the products’ abilities to both identify and report breaches within a 48-hour window. If the malware download/drop, or subsequently the execution (callback), was not detected within this timeframe, this would count as a failure.

While many products provide a means for identifying the malware on the drop/download, some products are better at detecting the execution (via callback). In this round of BDS testing, four vendors received a Recommended rating, and two vendors received a Caution rating.

The highest breach detection score achieved in the test was 99.1%. However, only two vendors did not miss evasions. Read the report to learn more.

So, back to our original question: Silver bullet or white elephant?