NetSecOPEN Fair Testing Has Arrived!

In a previous blog, I highlighted the merits of open standards-based assessments for today’s modern content-aware network security and inspection solutions. Understanding how well solutions work under a multitude of performance vectors can guide users with consistent empirical data on how well these solutions will work when under real-world conditions. NetSecOPEN has brought this to a reality via the IETF Benchmarking Methodology for Network Security Device Performance draft-02, allowing vendors and users alike to size up a solution’s capability via reliable, repeatable and transparent test constructs. The industry analyst firm HardenStance made a recent observation “The number of lawsuits between vendors and independent test firms in recent years points to how trust in third party security product testing is breaking down.” 1

No longer does the industry have to rely on inconsistent, “paid to play” private lab results that many times do not represent a solution’s real capability when used in mission critical environments... i.e. your network!

Well, all that work to create a new standards body has paid off. In mid-February of 2020, four major security solution vendors achieved NetSecOPEN certification. Cisco, Fortinet, Palo Alto Networks and SonicWall have all gained NetSecOPEN certification and have published full reports on specific firewalls via the NetSecOPEN standards methodologies. What does this mean?

At one level, this showcases these vendors desire to highlight how their solutions will work based on common and established configuration options that are in use by most organizations today. The NetSecOPEN requires specific device configuration specifications and right at the top of the certification reports, you can see how the security solution was configured for the obtained results:

With this information, you know specifically how many options for security inspection are active. Typically, the more inspection a device is expected to perform has a direct impact on overall performance and capacity.

At another level are the results themselves. They cover a wide range of common metrics that organizations can rely on to understand how a solution may fit for their needs, including HTTP and HTTPS/TLS throughput, transaction latencies and concurrent connection capacities.

And finally, there is the process. The assessment methodologies are fully transparent and open and the formal certification results are done by an independent lab, then the results are ratified by the NetSecOPEN standards body which includes exactly how the tests were run. This oversight ensures the results are accurate and there are no conflicts of interests in deriving the numbers. This provides real and reliable information to compare one solution to another.

Spirent is a founding member of NetSecOPEN and has been deeply involved in working with the consortium on defining and developing this standards-based approach to assessing modern security solutions. This consortium includes vendors, enterprises, test labs, and test tool providers - all working together to provide the market with new and accurate performance and security capabilities assessments that are purposefully impartial.

For more information on NetSecOPEN and to see these actual certification reports, go to https://www.netsecopen.org/certifications

Spirent solutions were heavily used in this initial set of NetSecOPEN certifications and all the test plans are available in the Spirent CyberFlood assessment platform, allowing users to model NetSecOPEN tests in their own labs.

1 HardenStance - A New Era in Trusted Network Security Testing White Paper Feb 2020