NSS LABS TEST POLICIES
FAQ: NSS Labs Mission and Tests
The NSS Labs Mission: Arm the world with the fact-based and objective information required to get secure and stay secure.
A key mechanism for accomplishing this mission is the NSS Labs group test motion. The group test motion relies on engagement with enterprises and security organizations around the world. Our active listening process includes one-to-one briefings, one-to-many briefings, and surveys. Our goal is to design test methodologies based on real-world enterprise use cases that help security organizations make educated decisions about purchasing and optimizing security infrastructure products and services.
Our expertise in threat and vulnerability research lends itself to addressing the harder problems of information security: exploits and malware protection, and all the ways attackers can circumvent security products. All of this can be confusing even to veteran IT security professionals so we focus on testing that helps to differentiate between products.
How do you determine what to test and how to conduct it?
NSS Labs determines our test roadmap based on market demand and real-world use cases. NSS Labs stays abreast of current threats and current solutions from a wide range of sources. We regularly hold briefings with enterprise users and vendors alike. Both constituents provide valuable feedback on what is important to them: the foundation of our test methodologies. All test methodologies are published and available prior to testing.
Are NSS Labs group tests “pay to play”?
No. Group tests are free to the vendors tested. Our policy is that no NSS Labs publication will ever be the result of a paid engagement, so if you see it in print, the test or research conducted was not the result of a paid engagement.
How are vendors selected and involved in the testing process?
For products that align to the enterprise use cases for a given methodology, NSS Labs determines inclusion of the vendor in a group test based on the following criteria:
- Market presence
- Enterprise and/or organization requests
- Innovative technology/solution (requires internal vetting for emerging vendors)
Vendors are notified upon consideration and upon formal selection. It is important to note that vendors do not get to choose whether or not they will be tested. We prefer that selected vendors come willingly and support the testing process, but if they decline, we may include their devices anyway, particularly if they have a significant market share and/or have gained broad visibility through bold claims to the marketplace. Under this scenario, products for testing are typically purchased, but may also be donated by interested parties, and we will conduct the testing independently.
How does NSS Labs make money from the group test motion?
NSS Labs fronts the cost of testing (which is often considerable). We are able to recover these costs in three primary ways:
- NSS Labs sells subscriptions to enterprises and other organizations that include access to our research, our test results, and our experts.
- NSS Labs sells single reports to individuals who want to see the results from a given test.
- NSS Labs offers marketing rights post-publication. While we don’t know how any given vendor will fare in a test, the tests reports do provide valuable differentiation for those that do well.
This approach allows us to remain objective and independent throughout the entire motion.
Does NSS Labs conduct product evaluations outside of the group test motion?
Yes. NSS Labs conducts private engagements for enterprises, government organizations, and vendors. Our policy explicitly dictates that any results from a private test cannot be shared externally by either NSS Labs or the party who engaged us. This means the results from a private engagement are just that, private, and they will never be shared publicly.