NSS Labs investigative reports draw from preliminary research on security technologies and are used to help our enterprise clients understand product capabilities. The research also informs our continuously evolving test methodologies. This blog accompanies a recent investigative report on the impact of code obfuscation and web-encoding techniques on the detection efficacy of next generation firewalls (NGFWs).
NGFWs are often marketed as the protectors of employee network traffic. Web traffic consumes a lot of bandwidth in the enterprise network and it is reasonable to assume that NGFWs are capable of providing comprehensive protection for it. However, the results of a recent NSS Labs investigation on NGFWs show that their accuracy in scanning JavaScript can be drastically impacted by common code obfuscation and HTTP/1.1 web-traffic encoding mechanisms.
Investigation Environment: The NSS test harness used a single benign JavaScript sample and a single malicious JavaScript sample served in web pages through each device under test. Fifteen separate code obfuscation techniques and four web traffic transfer-encoding mechanisms were applied to the scripts, and the devices were checked for scan results.
Test Results: All of the ten products investigated were impacted to some degree when exploits were transformed by one or more code obfuscation techniques. Implementation of web transport encoding mechanisms further impacted test results.
Not all products were affected to the same degree. Products with scanning engines that appeared to normalize the transport encodings prior to scanning appeared less affected by code obfuscation and encoding. However, normalization does impact throughput since it involves pre-processing (i.e., CPU impact), which could add latency if a product is running near its limit. Proper capacity planning should factor in these variables to limit impact to end users. (This is not true if an NGFW is incorrectly sized, or if it is an older generation device that has been updated with newer firmware and forced to handle new traffic.)
THE USE CASE: MOST OF TODAY’S WEB TRAFFIC
The modern web browsing experience is quite different to that experienced by web users even as little as five years ago. Complex content, background video, custom fonts, personalization (geography, time, etc.), and dynamic updates are intrinsic to the modern web page, which works to connect with users in order to retain them as visitors.
This browsing experience is made possible through advancements in multiple areas—browser applications, bandwidth, HTML, and the underlying HTTP protocol. Browser side-scripts are also integral to this experience. Downloaded from web servers, these scripts are run within the browser context and are used behind the scenes to activate much of the visual, audio, and automation mentioned previously. JavaScript (technically, ECMAScript) was formally released in 1999 and is the most prevalent of these—used by more than 94% of all websites.
Code obfuscation techniques are often seen as indicators of malicious intent, but they can play a legitimate role in the modern browsing experience. Minification, which is a form of obfuscation, reduces the size of HTML (including JavaScript) in order to reduce download requirements. It may also be implemented to slow the loss of custom code.
Code obfuscation and encoding techniques are commonplace; in fact, your navigation to this blog page likely involved several of these elements.
THE NEED TO UNDERSTAND PRODUCT LIMITS
The data from our investigation highlights a larger challenge within the industry: the lack of clearly-defined product capabilities. NGFWs are designed to scan and detect threats within network traffic, and it is reasonable for organizations to expect them to protect against web-delivered attacks. It is also reasonable for enterprises to expect that their NGFWs will be unaffected by content encoding transformations specified within the HTTP/1.1 RFC. However, our investigation results indicate that many NGFWs are in fact susceptible to these attacks and transformations, and this may undermine an organization’s confidence in its product—if it is being fooled by HTTP/1.1, what else is it missing?
Industry marketing teams often describe product capabilities in broad terms, which is misleading. To stay ahead of the competition, many marketing teams suggest capabilities that are nascent or not yet available. This obscures a product’s true capabilities—and limitations, which increases operational costs for security IT teams. Enterprise organizations require transparency to make informed purchasing decisions. In order to fully understand the capabilities of an NGFW, NSS recommends an organization rebuild all policies and comprehensively test them prior to deploying the product.
So, would you want to have an NGFW in your security architecture? More than 80% of US enterprises do, but the results of this investigation (and the results of the NSS Labs NGFW v8.0 Group Test) should be a wake-up call for NGFW product and product marketing teams.