No Vendor Excelled in NSS Labs-Defined Data Center Test Scenarios

AUSTIN, Texas – April 18, 2017 – NSS Labs, the global leader in operationalizing cybersecurity, today announced the results of its Data Center Firewall (DCFW) Group Test. Seven products from five vendors were tested. All seven products scored 100% in firewall policy enforcement testing, but no product demonstrated the lowest total cost of ownership (TCO) across all tested scenarios.

Enterprises store vast quantities of sensitive, business-critical information in modern data centers. Growth in the deployment of mobility and cloud computing technologies in data centers has transformed traffic patterns from a predominantly business-to-business (B2B) model with long duration connections commonly found in software-as-a-service (SaaS) applications to a business-to-consumer (B2C) model with short lived, high-volume connections, such as e-commerce transactions. When selecting DCFWs to support these use cases, enterprises look for products that enable maximum performance based on the traffic patterns they experience, without compromising security.

Enterprise-class data centers are designed to process multiple traffic mixes from potentially hundreds of thousands of users with high connection rates, concurrent connections, and performance requirements. Unlike next generation firewalls (NGFWs), which are part of a network’s perimeter defense and involve low-speed WAN links, DCFWs specialize in supporting the high bandwidth requirements of 40/100G top-of-rack and end-of-row deployments. NSS Labs recommends enterprises take into consideration the following key metrics regarding performance and cost when selecting DCFW products:

• TCO based on performance (TCO per Protected Mbps) – Throughput metrics are commonly used to benchmark products since they demonstrate how much information each product can process in cloud computing applications and secure transactions.
• TCO based on connection rate (TCO per Protected CPS per Mbps) – Connections per second (CPS) is critical for data centers that host applications with short connection durations at high transaction rates, such as e-commerce sites and high-volume database transactions.
• TCO based on concurrent connections (TCO per Protected Concurrency per Mbps) – Concurrent connections must be considered in scenarios where users are connected for longer durations, such as when using SaaS-based applications or media streaming.

In its group test, NSS Labs identified that the performance and capacity of the tested products varied extensively in each use case. Enterprises need to understand that one or more of these use cases should be factored into their decision processes. Other contributing factors should include firewall security through policy enforcement, stability and reliability.

Key findings from the test:

• No product excelled in all use cases
• 7 products achieved 100% effectiveness for firewall policy enforcement testing
• 2 products excelled in tests related to connection dynamics
• TCO based on performance (TCO per Protected Mbps) ranged from U.S. $0.90 to $2.57
• TCO based on connection rate (TCO per Protected CPS per Mbps) ranged from U.S. $4,347 to $38,201
• TCO based on concurrent connections (TCO per Protected Concurrency per Mbps) ranged from U.S. $43 to $467

“Selecting the right DCFW product is critical to business success,” said Jason Brvenik, Chief Technology Officer at NSS Labs. “Our DCFW Group Test results reveal that no single product excelled in all enterprise use cases. These results reinforce the need for enterprises to understand their current and near-term use cases and to define their requirements prior to evaluating products to determine which offering best meets their business, security, performance, and capacity requirements.”

The following products were tested:

• Cisco Systems Firepower 9300 v9.6.2.5 (one SM-36 security module)
• F5 i5600 v12.1.2 Build 0.0.248
• Fortinet FortiGate 1500D FortiOS v5.4.1 GA Build7386
• Fortinet FortiGate 3700D FortiOS v5.4.1 GA Build 7386
• Huawei Eudemon 8000E X16 v500R001C30
• Huawei USG9580 v500R001C30
• Juniper Networks SRX5400 15.1X49-D60

As with all NSS Labs group tests, there was no fee for participation. The test methodology applied is in the public domain to provide transparency and help enterprises understand the factors behind the results. The “no fee for participation” and “public domain” are part of NSS Labs’ commitment to provide empirical data and objective group test results that will enable security organizations to make educated decisions about purchasing and optimizing security infrastructure products and services.