| Karalon Traffic IQ Pro |
|
|
|
|
Traffic IQ Pro is essentially a packet replay tool, in a similar vein to the open source tcpreplay and Tomahawk tools, which can be used to verify the operation and detection capabilities of a typical IDS/IPS device (operating either in passive or in-line mode). It can also be used to validate non-proxy based packet filtering devices such as routers and firewalls. The product is installed on a PC with a minimum of two network cards – one connected to the internal interface of the product being tested, and the other connected to the external interface. The idea is that a capture file of network traffic (normal or malicious - such as might be created from tcpdump or Ethereal) is taken by Traffic IQ and divided into the two halves of the “conversation” - one consisting of those packets sent by the client, and the other those packets sent by the server. Each packet is then replayed in the correct sequence through the correct network card in order to arrive at the appropriate interface of the sensor. Traffic IQ can be configured to handle transmissions with or without default gateways, and with or without NAT (internal and external) as required. As packets are placed on the wire, the user has the option of overwriting source/destination IP addresses, source/destination port numbers, source/destination MAC addresses and TTL values. This allows the user to replay capture files from third parties using IP address ranges which match the test network exactly, although any combination of these can be left as the originally-captured values if preferred. As these values are overwritten, Traffic IQ checks the HTTP headers and FTP commands to ensure that any original IP addresses and/or port numbers located there (such as HTTP referrer field, FTP port command, etc.) are also replaced to match the changed addresses. An Advanced Settings tab allows this behaviour to be suppressed, leaving the original values untouched. Naturally, since a significant amount of data is being changed in each packet, all sequence numbers and checksums are recalculated using the new data before the packets are sent on their way. Multiple sessions contained within a single capture file are handled correctly, with intelligent replacement of the IP addresses for each session. During the replay operation, the sensor will see the correct SYN on the correct interface, followed by the SYN ACK on the opposite interface, followed by the ACK on the first interface, and so on throughout the capture file. Thus, in theory, the sensor will see the attack as it was originally played across the wire. This does work most of the time, but it is important to realise that Traffic IQ does not actually implement a complete replacement stack, and thus with certain exploits or with badly-crafted capture files things can go wrong, resulting in invalid or out-of-sequence packets on the wire. When this happens, depending on where it happens in the replay, the sensor may quite legitimately ignore the “exploit” and no alert will be raised. This occasionally makes it difficult to determine if the fault lies with Traffic IQ or the IPS/IDS device being tested. Traffic IQ, therefore, is no panacea - you still need a reasonable knowledge of vulnerabilities, exploits, and what a “good” capture file should look like in order to get the most out of a product like this. In this respect, however, it is not alone, and suffers from the same shortcomings as all other replay tools. It is thus not applicable for all your exploit testing (it cannot handle badly fragmented/segmented traffic, for example). However, for those who prefer a Windows-based GUI to the Unix command line, Traffic IQ offers a much more usable user interface for basic attack recognition testing with well-crafted capture files (such as those included in the Traffic IQ exploit library) than typical open source offerings. The open source offerings do provide their own advantages, however. Both Tomahawk and tcpreplay, for example, provide much higher transmission rates than is possible with Traffic IQ running on Windows. On the other hand, open source tools do not come with a well-stocked, ready-to-run, library of exploits. Thus we would recommend using Traffic IQ as just one of several such tools when performing IDS/IPS testing. At the end of the day, with some “unusual” exploits which do not play nicely with capture files and replay tools (and especially when testing certain evasion techniques involving heavy fragmentation/segmentation) there is simply no substitute for running the original exploit (either using actual exploit code, or via tools such as Metasploit Framework or CORE IMPACT). This is what NSS does in its own tests, running a mixture of replay tools such as Traffic IQ Pro, tcpreplay and Tomahawk with our own library of exploits, together with tools such as Metasploit and CORE IMPACT, and often resorting to using live exploit code where necessary. There is no one-size-fits-all shortcut or magic bullet approach to testing signature recognition capabilities. It is not easy to do well - tools such as Traffic IQ simply make it easier. In addition to replaying individual capture files, Traffic IQ allows the administrator to create Groups and Traffic Scan Lists. These are collections of exploits which can be replayed individually or as collections, but whereas Groups are quicker to create, they utilise the same IP address, port numbers and direction (client-to-server or server-to-client) for all replayed captures, which is not always convenient. Traffic Scan Lists, however, allow the user to define unique settings (IP address, port, direction, time limit, expected result, number of repeats, location on disk) for each capture file in the list. This not only ensures that different IP quads are used for each exploit run, but that the resultant alerts are easier to spot in the IDS/IPS logs. Vendor descriptions of alerts rarely match up to each other, to the CVE reference, or to Karalon’s own descriptions, and thus it makes life much easier to be able to determine that the alert entered in the log files from address 10.10.107.106 was actually generated by a particular exploit in the Traffic Scan List. An extensive right-click menu is available when editing Traffic Scan Lists, allowing bulk changes of IP addresses, ports, traffic direction, expected results, and so on, as well as applying auto-increments on port numbers and specified quads of the IP addresses. This is a very powerful feature which makes it extremely easy to create large Traffic Scan Lists from scratch. Once created, these lists can be saved for later recall and/or edit. Another excellent feature is the Traffic Editor, which allows the user to edit any capture file which has been imported into Traffic IQ. The user is provided with an Ethereal-like display of the packets, and can type directly into the packet buffer to make changes. Selecting the fields in the hierarchical packet menu as required highlights the necessary bytes in the packet buffer to make changes easier. A search and replace/fill option provides the means to make mass changes within a packet or across all packets - particularly useful for foiling simple pattern-based, exploit-specific signatures by changing the overflow buffer fill character from all ‘A’s to a random mix of characters, for example. Obviously, checksums are recalculated where necessary as the capture file is saved to keep the traffic “legitimate”. As the exploits in a Traffic Scan List are run, a count of the number which succeeded (i.e. where all the packets made it through the device) or failed (i.e. the exploit was blocked successfully by the device under test) is displayed on screen. A summary of which exploits succeeded and which failed is then available via the Reports option. By switching between Audit Logging and Packet Logging in the Report Options, it is not only possible to see which exploits failed, but which packets within the capture file were blocked by the device under test. This is a very useful feature in determining whether an IPS blocked an exploit in time to prevent the actual payload from being delivered. The Import feature allows the user to create Traffic IQ files from standard PCAPs. Nothing of note is done during the conversion process (a header is added, a checksum is created, and the PCAP is encrypted), but it is necessary - standard PCAPs cannot be run within Traffic IQ. During the import process, a description of the capture file can be added (NSS usually includes the CVE reference, BID reference and exploit description, for example), and this is displayed whenever a capture file is selected at any point in the Traffic IQ Pro interface. Multiple attack libraries can be created and placed in separate subdirectories, from where they can be selected at run-time within Traffic IQ Pro. In this way, it is easy to switch between different libraries - NSS uses this capability to switch between the different libraries used for each of our group test reports, for example. Many users will be satisfied using the pre-packed exploit library provided by Karalon as part of the product, in which case they would never have to worry about creating and importing their own. Most IDS/IPS products will have signatures for the majority of the Karalon exploits these days, and so as a tool to verify that your IDS/IPS appliance is detecting, logging and alerting correctly, Traffic IQ is incredibly useful. However, even for those who shun the built-in library and need to create capture files from scratch from live exploits, Traffic IQ offers some key advantages - mainly in the area of ease of use and repeatability. Some of the exploits NSS runs as part of the standard IDS/IPS test suite are very complex to set up - perhaps requiring unusual combinations of operating systems and services - and it can be a painful process to recreate them over and over again for each product tested. Using a packet capture tool (such as Ethereal or tcpdump) and the import capability of Traffic IQ, it is now only necessary to run the exploits once, before converting them to Traffic IQ traffic files. After that, they can be replayed quickly and easily (via a user-friendly GUI or command line interface) time and time again, and it is certain that every device tested sees exactly that same traffic as the rest. NSS also receives useful trace files from our vulnerability research partner, Assurent. The problem with these, of course, is that the IP addresses could be anything. Often they are unusable as they stand since the addresses used might fall outside the range allowed by the license key of the product being tested. Traffic IQ gives us the opportunity to use those trace files since, once they are imported, the IP and MAC addresses can be replaced as the attacks are run. However, the product does have a number of disadvantages (which will vary in importance or relevance depending on the user): it cannot handle heavily fragmented/segmented traffic; badly constructed capture files can cause “misbehaviour” when replaying them (though as the product becomes more and more sophisticated this is an increasingly rare occurrence); it costs money (open source tools are free); and the built-in attack library has an excess of Backdoors and Trojans (many of them very obscure). The library is constantly improving, however, with many newer attacks taken from the Metasploit Framework, obviating the need for users to obtain Metasploit and run the exploits against live servers themselves. The biggest drawback with Traffic IQ for major testing projects is the very availability of its exploit library. It is almost impossible to use the built-in library on its own as a test for signature coverage in a competitive environment (whether it is an NSS group test or your own internal bake-off) because every vendor has access to it, and most of them will ensure that they have signatures to cover it. Thus, as a comparison between products, it serves little purpose (unless you come across a “rogue” product that actually cannot detect most of the Karalon exploits, which would be odd in itself). This is why NSS, as a testing organisation, cannot rely on the Karalon library entirely, and is thus forced to produce custom own capture files for each group test. Note that this is not a fault of the library itself, just the way it is being used. However, as proof that your own device is working correctly following installation, policy configuration and/or product updates, it serves a valuable purpose. If a test tool provider could produce a tool like this and produce regular updates to the exploit library in the manner of the average Anti Virus software vendor, then administrators will finally have the capability to audit and verify the efficacy of the latest signature pack from their IDS/IPS vendor, and that test tool will really come into its own. Updates to the Karalon library are appearing more frequently at the time of writing than under previous releases. Overall, the Karalon attack library is very useful for testing against both older and more recent exploits, and it also provides a wide range of “normal” traffic files (HTTP, SSH, FTP, and so on). Regardless of whether you are using the built-in library or custom attack files, Traffic IQ Pro is an extremely useful tool for replaying these across your network in a consistent and flexible manner, making it far easier than it has ever been to test IDS/IPS products for effective attack coverage. |
