Press

NSS Labs Announces Next Generation Intrusion Prevention Systems Test Results

October 13, 2016
2,577,402 suspicious URLs yielded over 2,400 drive-by exploits used by threat actors in active campaigns.
Two products received Caution ratings.

AUSTIN, Texas – October 13, 2016 – NSS Labs, Inc., the world’s leading cyber security product research, testing, and advisory company, today released the results from its Next Generation Intrusion Prevention Systems (NGIPS) group test. Eight of the leading NGIPS vendors were examined for security effectiveness, performance, and total cost of ownership: Check Point, Cisco, Forcepoint, Fortinet, IBM, McAfee/Intel, Palo Alto Networks and Trend Micro.

Cyber-criminals have become more aggressive over the past years, increasingly targeting corporate assets. Vulnerability disclosures in widely deployed operating systems and applications are a growing problem. Designed to identify and block attacks, a good NGIPS can provide temporary protection from the immediate need to patch vulnerable systems. An NGIPS must catch sophisticated attacks without producing false positives or degrading network performance.

Using the NSS NGIPS Test Methodology v2.0, NSS Labs performed the most comprehensive NGIPS test to date. Products were tested from June 2016 through September 2016, with a live component of drive-by attacks from August 28th 2016 through September 26th 2016, using NSS Labs’ Cyber Advanced Warning System™ (CAWS).

  • 1,986 exploits were deployed from NSS Labs’ extensive exploit library.
  • 120 evasion techniques were utilized in the test.
  • 2,577,402 suspicious URLs yielded more than 2,400 drive-by exploits used by threat actors in active campaigns at the time of testing; the largest live test ever conducted.
  • Active drive-by exploits were tested for up to three days resulting in 48,488 discrete test cases across over 7,000 live victim machines.

Test highlights include:

  • Two (2) products received a Caution rating (25% of products tested)
  • Two (2) products received a Neutral rating.
  • Four (4) products achieved a Recommended rating
  • Security Effectiveness ranged from 24.9% to 99.9%
  • One (1) product leaked attacks under heavy traffic loads when state preservation was exceeded.
  • Three (3) products were rated below their stated throughput; the other five (5) were rated at or above their stated throughput.
  • There was no direct correlation between price and effectiveness; more expensive products did not always do better. Total Cost of Ownership (TCO) per Protected Megabit per Second ranged from US$8 to US$27.

“Next Generation Intrusion Prevention Systems are designed to protect against a new generation of threats that move faster and are more evasive than ever before,” said Thomas Skybakmoen, Distinguished Research Director at NSS Labs. “Exploit block rates, consistency of protection over time, and false positive rates are critical metrics for enterprises to consider when purchasing an NGIPS.”

The following Vendors and Products Tested during this test:

  • Check Point Software Technologies, Ltd. 13800 Next Generation Firewall Appliance vR77.20
  • Cisco FirePOWER 8350 v6.0.1
  • Forcepoint Stonesoft Next Generation Firewall 3301 v6.0.2
  • Fortinet FortiGate 3000D v5.4.0
  • IBM Security Network Protection XGS 7100 v5.3.2.1
  • Intel Security McAfee Network Security Platform NS9100 v8.2.5.158
  • Palo Alto Networks PA-7050 v7.0.4
  • Trend Micro TippingPoint 7500NX v3.8.4.4525