Press

NSS Labs Reveals First Security Value Map™ for Breach Detection

April 2, 2014

Four of Six Leading Vendors Receive Coveted NSS “Recommended” Rating

AUSTIN, Texas – April 2, 2014 − NSS Labs today released the results and analysis from its first Breach Detection Systems (BDS) Comparative Analysis and Security Value Map™ which evaluated six of the leading BDS vendors – AhnLab, Fidelis, FireEye, Fortinet, Sourcefire (Cisco) and Trend Micro – for security effectiveness, performance, and total cost of ownership. In rigorous, independent testing, NSS deployed vendors’ products in simulated enterprise environments and evaluated their ability to detect a wide range of malware and exploits attackers use in malicious Web, e-mail and other delivery methods to evade security products, gain footholds in networks and compromise sensitive information. With the entry of several vendors into the BDS market and given the ongoing pace of large, high-profile data breaches affecting retailers and other organizations, third-party comparative testing is critical for helping buyers identify the key criteria when selecting a BDS solution and whether vendors in this quickly evolving space are delivering on their capabilities and claims.

View the NSS Labs Breach Detection Systems (BDS) Security Value Map™

NSS’ key findings include:

  • Four of six products tested achieved over 95% in overall security effectiveness: Overall security effectiveness ranged from 94.5% to 99.1% with Trend Micro scoring the highest and four of the six over 95%. The ability to keep false positives to a minimum is as important as detection rate alone, so it is notable that five of the six also received a 0% false positive rate.
  • Money Doesn’t Always Buy the Best Security: Total Cost of Ownership per Protected-Mbps ranged from $231 to $468 with the highest priced solution, AhnLab MDS, receiving the lowest security effectiveness rating. Conversely, Sourcefire (Cisco) had the lowest TCO and also received one of the highest security effectiveness ratings.
  • All BDS Solutions Performed At or Above Vendor Throughput Claims: Five of the six products tested achieved their vendor-stated performance rates of 1,000 Mbps in testing and one exceeded its 250 Mbps vendor-stated throughput at 667 Mbps. There were significant differences in the maximum number of TCP connections and concurrent connections allowed, so enterprises evaluating a BDS should consider those results as well.

Commentary: NSS Labs Chief Executive Officer Vikram Phatak

“Breach Detection Systems are one of the most rapidly evolving security technologies out there today and with that comes a lot of marketing hype and vendor claims. We are excited to be the first to test and provide empirical insight into the performance and capabilities of the leading vendors in this emerging market,” said Vikram Phatak, CEO at NSS Labs. “With several key vendors entering the BDS market, the BDS SVM results are an excellent example of why independent testing is so important. They provide objective facts based upon empirical data, allowing executives to make educated purchasing decisions.”

The products covered in this test were:

  • AhnLab MDS
  • Fidelis XPS Direct 1000
  • FireEye Web MPS 4310 and Email MPS 5300
  • Fortinet FortiSandbox 3000D
  • Sourcefire (Cisco) Advanced Malware Protection
  • Trend Micro Deep Discovery Inspector Model 1000

NSS Labs did not receive any compensation in return for vendor participation; All testing and research was conducted free of charge.