| IPS Exploit Selection Criteria |
|
NSS Labs - in conjunction with its security research partner Assurent Technologies - spends a great deal of time carefully selecting the test cases used in IPS testing. Because it is important that vendors yet to be tested are not given an advantage in the detection tests, it is NSS policy not to disclose the definitive list of exploits/CVE references which form the test suite. This is because NSS believes that it is not enough to simply select a group of off-the-shelf testing tools and use whatever outdated exploits are contained within the tool libraries - this type of test is easy for vendors to prepare for and provides no useful information to the end-user or prospective purchaser. Instead, NSS carefully selects test cases which fall into different categories of severity based on: whether the exploit will provide root/administrator access on a widely deployed operating system or application; whether it imposes a DOS condition with no risk of system compromise; whether it is against a system which is not widely deployed or is not Internet-facing; whether it is purely a reconnaissance technique designed to gather information for a subsequent attack attempt; and so on. We then test each device with the default policy or recommended settings provided by the device out of the box. This gives users a clear indication of how any IPS device will perform should they simply remove it from the box, install it and switch it on. We then tune the policy to turn on audit/informational signatures, and remove any which are causing false positive alerts, and we test again. This gives users a clear indication of how a properly configured device will perform in their network. At no point in this process is the vendor provided with detailed information on the test cases nor are they given the opportunity to update their device with new signatures - the idea behind this particular test is to determine how accurate is a device out of the box rather than determine the speed of response of the vendor's vulnerability research and signature writing operation (NSS operates a separate service for that purpose). The majority of exploits for Test 1.1.1, 1.1.2 and 1.1.3 fall within the last 12 months, with some falling in the 12-24 month range. None are older than 24 months EXCEPT where serious exploits are to be found in commonly available tools such as Metasploit or Core Impact. Some older exploits are to be found in Test 1.1.4 in order to determine which products need to “clean out” older signatures at regular intervals in order to maintain performance levels. Test 1.1.5 includes many reconnaissance techniques which have CVE references allocated which are several years old. However, these are all standard recon techniques - not exploits, per se, but common techniques for footprinting, scanning and enumeration - as detailed in any good “hacker’s handbook” or ethical hacking training course, and are therefore considered “timeless”. Some IPS vendors may consider these as IDS signatures, since they are not actual threats. However, since many IPS vendors do include these signatures (albeit disabled by default) there is no minimum detection requirement, but the section is included to highlight those products which are able to detect and block such attempts. The DOS/DDOS attacks in Test 1.1.6 are live SYN flood attacks from a single source IP and from multiple source IPs generated by high performance attack tools from Spirent Technologies. Test 1.1.7 includes some of the most common Peer-to-Peer traffic seen on today’s corporate networks. This section is currently optional. Test 1.1.8 contains common Adware/Spyware traffic taken from the network both when the malicious application is being downloaded and when the application has been installed and is “calling home”. The test suite concentrates on those applications considered to pose a serious threat to network security, such as those which offer remote control facilities or attempt to transmit private data back to the source. We consider the test result to be positive should the DUT detect any of this traffic at any point during the life-cycle of the application. This section is currently optional. Test 1.1.9 contains recent Server-to-Client exploits which typically provide remote access to the client PC once executed. Obviously, Microsoft’s Internet Explorer features heavily in this test suite, though we will also include exploits for other common client applications such as Outlook, RealPlayer, telnet, QuickTime, Microsoft Office, and so on. This section is currently optional. A wide range of operating systems are represented in the NSS detection tests, including: • Microsoft Windows (all versions (inc. NT4), ages and patch levels) • Sun Solaris • Linux (various versions/vendors) • BSD • OS X A wide range of applications/protocols are also represented, including: • HTTP • FTP • DNS • IMAP • POP3 • SMTP • DHCP • MS-RPC • NetBIOS/SMB • WebDAV • Microsoft DTC/CDO • Microsoft Exchange Server • Microsoft Outlook Web Access (OWA) • Microsoft IIS (various versions) • Microsoft FrontPage (including Server Extensions) • Microsoft Index Server • WINS • Microsoft SQL Server • Microsoft ISA Server • Oracle Database Server/Application Server • Apache • Sendmail • Snort • Squid • MySQL • Novell eDirectory • HP OpenView • IBM WebSphere • Kerberos • BrightStor ARCserve • Veritas NetBackup • Veritas Backup Exec • IBM Lotus Notes • Sybase EAServer • A range of common Web browsers (S2C) • Outlook (S2C) • RealPlayer (S2C) • QuickTime (S2C) • Microsoft Office (S2C) Please note that the aforementioned list is not intended to be exhaustive, but an indication of the types of exploit test cases the DUT is likely to face. |
