nsslabs.com

Definition

The ongoing threat from fast spreading zero-day Internet worms, scanning reconnaissance from hostile entities, and distributed denial of service attacks ranging from simple bot networks to organized crime; has necessitated the requirement for Attack Mitigators in many Internet services environments.  Unlike signature based IPS that are designed to deal with known exploit vectors, Attack Mitigators are designed to deal with the unknown anomalies and irregularities that characterize the other end of the attack spectrum.  Attack Mitigators utilize several rate and anomaly based detection methods to identify friend from foe.  Once identified, Attack Mitigators use a series of mitigation technologies to prevent the attackers’ hostile traffic from interfering with normal users’ legitimate traffic.

The NSS Labs Attack Mitigator certification attests to the abilities of a product to properly identify and mitigate hostile attacker traffic while maintaining sustained loads of legitimate user traffic.  

Attack Mitigator Criteria

At the most basic level an Attack Mitigator product must identify hostile attackers and mitigate their undesirable network traffic while allowing legitimate user traffic to pass.  NSS Labs Attack Mitigator testing focuses on the complex interaction between these fundamental capabilities. 

First, the Attack Mitigator is tested to detect and block a series of baseline attack and scanning methods.  Second, the product’s performance is measured using a combination of protocols that gage the maximum performance under ideal conditions.  Third and final, the product’s performance is re-measured while under hostile attack from the baseline attack and scanning methods.  This third stage measures the overall effectiveness of the Attack Mitigator to block attackers’ hostile traffic while continuing to allow the normal users’ legitimate traffic.  Impact on the normal users’ legitimate traffic is rigorously measured and fully documented.

Baseline Capabilities

Using known scanning and attack methods, NSS Labs validates the ability of the product to classify and properly identify hostile attackers against exposed IP based services.  The target hosts are placed inline behind the Attack Mitigator and subject to the following:

Ping Sweep – Sequential and pseudorandom ICMP/Ping scanning at varying rates from a single source targeting multiple protected hosts.  

Port Scanning – TCP port scanning based a variety of SYN rates from a single host.

Obfuscated Port Scanning – TCP port scanning based on non-SYN mechanisms such as ACK and FIN from a limited subset of hosts.  Scans are also initiated from pseudorandom sources within the same relative addressing space as if an attacker was attacking from several hosts on the same IP subnet.

Denial of Service – Both stateless and stateful DoS are generated from a limited number of attackers.  Stateless attacks contain TCP and UDP packet blasting.  Stateful attacks comprise of protocol specific DoS attacks that target specific services.  Several of these DoS attacks have well known CVE and other public references.

Distributed Denial of Service – DDOS is the expansion of the DoS testing suite across incremental quantities of attackers.  Typical DDoS are sourced from tens of thousands of attackers.  Large DDoS are sourced from millions of attackers.

Random Protocol Mutations – Random protocol fuzzing and mutation are generated from a small subset of attackers.  Many products and services exhibit instabilities when exposed to unexpected / random protocol content.  An Attack Mitigator should block these random protocols from reaching the protected hosts and services.

Multiple Attack Correlation – More sophisticated attackers may attempt to hide their activities by probing diverse entry points into a protected environment.  By probing the European perimeter then the North American perimeter with limited scans, an attacker may attempt to evade detection.  To combat this, some Attack Mitigators support a centralized correlation engine that mitigates these scans and blocks the attacker across all perimeters.  For Attack Mitigators that support this capability, NSS Labs provides a series of sophisticated multisite attacks across diverse entry points. 

Performance

Performance of an Attack Mitigator must be measured carefully using realistic traffic conditions and protocols.  Typical benchmark traffic from a limited number of sources at high connection or transaction rates is both unrealistic of the real world and may inadvertently trigger the Attack Mitigator’s rate based thresholds for DoS conditions.  NSS Labs has methodically constructed real world traffic topologies in our lab environment that exhibit characteristics of common web sites and IP based services.  Using this traffic mix we can accurately measure:

Packets per Second – Bidirectional RTSP streams of varying packet sizes are requested from diverse sources analogous to bidirectional video traffic. 

Connections per Second – TCP connections per second with minimal HTTP content and one HTTP connection per second focusing on connection dynamics typical with Internet search engines.

Maximum Concurrent Connections – Concurrent TCP connections using long server processing times typical of many trending and reporting environments.

Transactions per Second – HTTP transactions per second with multiple HTML objects requested across few TCP connections typical of web sites containing many style sheets, graphics, and subsequent objects within a single web page.

Effective Throughput – Internet traffic mix of common web sites with similar connection rates and content sizes.

Unidirectional latency through the Attack Mitigator while processing RTSP packet streams.

The latency and throughput of an Attack Mitigator must be on par with other equipment in the network on which it is deployed.  For example, an in-line Attack Mitigator must strive to perform much more like an Ethernet switch than a typical passive security device, especially when it is necessary to install more than one appliance in the same data path. 

Effectiveness

The effectiveness of an Attack Mitigator requires merging the capabilities and performance dynamics into a series of test cases that accurately measure the overall system impact on legitimate user traffic while mitigating attacker traffic.  NSS Labs tests this overall effectiveness by using the Effective Throughput content from the previous performance testing at varying percentages of the maximum manufacturer rated throughput while subjecting the Attack Mitigator to the Baseline Capabilities tests.  This multiple dimensional testing results in the following:

This twofold approach measures both the effectiveness of the Attack Mitigator at detection and mitigation (while supporting the desired throughput levels) and any impact on the legitimate user traffic.

Management

After quantitatively evaluating the performance and security effectiveness of the Attack Mitigator, NSS Labs qualitatively evaluates the features and usability of the product. This evaluation provides the reader with valuable insight into product features, how easy it is to configure the device and perform common, day-to-day operations with the management interfaces. Areas evaluated include configuration, policy management, alert handling, and reporting.

Certified Attack Mitigator Products