Testing Challenges and Strategies for Successful SASE Deployments


SASE emerged in 2019 as a visionary security framework where security functions are hosted in the cloud to address security challenges resulting from the rapid transition to a remote, hybrid workforce – but not without challenges. Learn how to overcome these challenges and operationalize SASE network architectures and services running on these networks.

Secure Access Service Edge (SASE) converges networking and security domains into a distributed cloud environment, where it connects and secures any IT resource—physical, virtual, and mobile—with unified management and policy. This dynamic, policy-based approach delivers the agility, elastic scaling, and resiliency digital enterprises require, achieving significant advantages over VPNs deployed in a traditional centralized data center. As technologies converge, there are new challenges to overcome on the road to delivering the hybrid distributed systems that intelligent operations demand.

New challenges on the road to distributed network security

Key operational challenges for SASE deployments

1. SASE assurance

Managed Service Providers (MSP) are obligated to deliver comprehensive service-level agreements (SLA) to their enterprise end-users. However, unlike network/infrastructure SLAs, the SASE KPIs have not been standardized, varying widely for each specific service and application. At the same time, SASE testing standards are just beginning to emerge. Establishing a methodology to validate SASE end-to-end behavior, especially considering the many proprietary SASE-based CNFs, NFVI variations, and lack of purpose-built tools, can be expensive and time-consuming.

2. Network functions and applications assurance

To effectively deliver SASE end-to-end service-level management, MSPs must validate each network function that is deployed in the SASE edge cloud. The complexity is exacerbated by the many proprietary network functions, each with its own API and management tool, and the lack of a common assurance methodology. Again, SASE testing standards do not exist yet, and at this early stage there are no purpose-built tools, compelling MSPs and their partners to rely upon home-grown testing tools that are unable to scale.

3. SASE service applications behavior

An important factor in the selection of cloud security controls such as next-generation firewalls (NGFW), web application firewalls (WAF), or secure web gateways (SWG) is the footprint, scalability, and robustness running in various cloud environments. Validating the efficacy of the security controls is essential but requires specialized expertise and realistic emulation of legitimate and malicious traffic profiles to validate effectiveness and performance in conjunction with finding the optimum balance between quality of experience (QoE) and security effectiveness.

4. Security policy and performance assessment

Security rule set validation is imperative for successful rollout and policy configuration for SASE environments. The primary challenge is the ever-changing threat landscape. Assessment of security rule sets needs to be performed on a continuous basis, not at a specific instance in time, to account for new threats and vulnerabilities, evolving policies and network configuration and inventory changes.

5. Zero Trust Network Access (ZTNA) behavior

ZTNA relies on trust brokers to grant access based on identity, policy, and context (vs network connections). MSPs must be able to validate ZTNA elements for scale and sustainable access request rate, while ensuring that the policy criteria are continuously enforced by security controls like NGFW and data loss prevention (DLP) for application and data access. Lack of ZTNA standardization has resulted in proprietary products or services with varying capabilities, making it harder to quantify and contrast different solution options.

Given these various challenges, how do we operationalize multi-domain, distributed, and dynamic SASE deployments? In the next section, we will discuss the principles and test methodologies we leverage to enable our customers to overcome the above-mentioned challenges.

Modern SASE, secured SD-WAN and application security architecture

Testing strategies for operationalizing SASE deployments

1. Test every deployment environment

Considering SASE architectures are multi-domain, hybrid, and distributed, SASE deployments need to be validated end to end – from the remote users and branches through the SASE POPs (points of presence) to end-application servers. SASE managed service offerings may be connecting over many IPS networks and different public and private clouds. Therefore, it is important to characterize the latency and performance (throughput, transactions per second) profiles of these networks and verify whether the SASE architecture is improving or hindering overall performance and latency as it secures these networks.

These principles apply whether you are creating 5G Core or Metro Edge, operating a quality assurance lab, or integrating with CI/CD/CT developer toolchains. Holistic testing and assurance capabilities are essential, regardless of what the architecture is based on, VMs, containers, or bare metal.

2. Select traffic patterns and KPIs according to business needs

Selecting appropriate SASE test traffic patterns that are representative of your network traffic and applications will ensure that performance is characterized under realistic conditions, avoiding the false confidence that comes with simple traffic patterns offered by open-source tools. Testing all security components of the SASE framework, including the network underlay, cloud infrastructure services, and business applications, requires multiple KPIs for all technology and services levels.

Quality of Experience (QoE) is the best unit of measure because it directly indicates end-user satisfaction. It is based on performance, detection of errors, and variability for SSL/TLS based services and overall transactional latency. This metric is also optimal for altering the underlays that application flows will forward across the SASE environment. In addition, bandwidth/throughput, concurrent users and connections, connection and rate are good baseline metrics to help right-size infrastructure to business needs and continuously baseline as part of the CI/CD/CT practice, for a measured approach towards operationalizing the change management process.

3. Identify security and performance vulnerabilities

Besides validating realistic application traffic profile performance and QoE, there is also a need to realistically model threat vectors to validate the security efficacy of the SASE security stack with the latest threat vectors. For this, it is essential to have access to a constantly evolving library of threat content covering vulnerabilities, malware, exploits, and ensuring alignment to security industry framework such as MITRE ATT&CK, to improve the organization’s security posture and bridge gaps in SASE deployments.

To fully characterize the security effectiveness, it is important to emulate hacker-like behavior by using evasion or obfuscation techniques, including cloaking threat vectors in TLS. There is also a need to characterize the impact of security policies on end-user experience by emulating legitimate application traffic at scale in conjunction with malicious threat vectors. If security controls and polices are impacting business-critical activities, the organization will look for ways to bypass those controls, which could adversely affect its security posture.

4. Characterize the impact of Zero Trust (ZT) architecture

Since Zero Trust is a critical component of SASE architecture, it is vital to characterize the scalability of the Zero Trust architecture in terms of the authentication rate for concurrent users that the identity and access management (IAM) can support. In addition, characterizing the effectiveness and impact of ZT policies such as micro-segmentation, lateral threat movement and DLP on performance and end-user QoE is also important.

The need for intelligent automation, reporting and data-driven predictions

Validating multiple levels of cloud and services infrastructure for complex use cases—such as 5G low-latency slicing or 360⁰ security policy enablement for distributed environments—requires an additional level of intelligence, as well as comprehensive automation, reporting and correlated data rather than partial, incomplete information.

Business impact analysis is essential for understanding how new policies or service disturbances will affect your end-customers and SLAs. Root cause analysis supplemented by anomaly detection will provide in-depth analysis, not only for historical data, but also to expose future issues and performance limitations. In short, the massive amount of data generated by hundreds, or thousands of applications requires a profound understanding of the impact of security policies on performance, and predictive analytics to accurately forecast future behavior and potential impact on security policies.

The agile way in which SASE networks and policies are evolving requires validation that is continuous and integrated into operationalizing change management practices to best manage security and performance drifts.

Learn more about the requirements for effective SASE and Zero Trust validation, related use cases and testing strategies右箭头图标




Oleksandr Dmytriiev


Oleksandr Dmytriiev现任思博伦的SASE/SD-WAN解决方案高级产品经理。他当前的职责是管理融合了安全、网络基础设施和云领域多种技术和产品的下一代解决方案。在加入思博伦之前,Oleksandr曾在多家电信软件厂商和大型企业就职,负责管理诸多软件产品线的产品管理团队,业务领域涉及SD-WAN、OSS、BS云和机器学习平台等。如欲联络Oleksandr,请关注他的LinkedIn:https://www.linkedin.com/in/odmytriiev/