Ensuring the best possible user experience when launching new differentiated 5G services is an essential component of any service provider’s (SP) ongoing operational strategy. At the time of any service launch, SPs must know the answers to these critical questions:
What is the overall user experience on our network versus competitors?
Have we been successful in our differentiation strategy?
What are our greatest strengths?
What top service areas do we need to improve?
How is our spectrum strategy progressing compared to our competitors?
We want to be the best at augmented reality (AR), how do we get there?
Will our large business customers be satisfied with our network performance?
How do we prove our offer (e.g., mobile edge computing – MEC – latency) is better?
Challenges persist in answering these questions with satisfaction and confident authority. In my earlier post, I described the need for benchmarking services to validate marketing claims and identify critical issues for service optimization in networks, especially as latency, cloud gaming experience, OTT video, and the impact of spectrum decisions are quickly rising in prominence for 5G benchmarking.
SPs typically conduct benchmarking testing twice a year to validate that their networks are delivering quality end-user experiences in their existing services as originally planned. But whenever new services are being developed, it’s a larger story. Assuring these services work as intended involves a prolonged and ongoing testing strategy to ensure quality of experience (QoE) and enduring end-user satisfaction.
In some cases, benchmarking leads to a more extended dive into a network’s offerings, with a testing strategy that delivers an ongoing set of data for continuous improvement of an SP’s network. In other cases, SPs know from the start that they need to implement a mobile service launch validation program for their 5G network, especially their strategy involves continuous improvements and new services to the network.
Strategies for service launch validation for 5G networks
First: measure what matters. Zeroing in on testing the essential components of an overall service is critical in assessing the comprehensive quality of data, video, voice, 5G, cloud gaming and location accuracy in a network’s services delivery.
In adopting a state-of-the-art 5G testing strategy, it’s important to understand that the traditional approach used for 4G is not suitable for 5G, which encompasses many more technologies. 5G offers new levels of agility bandwidth performance, latency, and average speed. The testing approach must also account for multiple handoffs between 2G, 3G, 4G, and 5G bands, as well as Wi-Fi and Wi-Fi 6, and multiple bands in the RAN (mmWave, mid-band, low-band), mobile edge computing (MEC), and the evolution to 5G Standalone (SA) must be incorporated in any service validation program in 5G networks.
From there, with a primary focus on the user experience (UE), having the KPIs in the various categories must be clearly defined, which include:
5G: 5G connection rate / availability; 5G reliability
HTTP/FTP Download/Upload: Upload / Download mean throughput rates; Reliability: cycles (aborted, failed, passed)
Bandwidth Download/Upload: Upload / Download mean throughput rates; Reliability: cycles (aborted, failed, passed); Bandwidth score
UDP Download/Upload: Reliability: one-way latency; cycles (aborted, failed, passed); Jitter; Instantaneous, mean and percentage of ideal throughput
Web Browsing: Reliability: cycles (aborted, failed, passed); Page load time
Ping: Reliability: cycles (aborted, failed, passed); Latency: Round trip time; Packet loss percentage
Video KPIs: Video MOS; Freezing and buffering rate; Packet Loss Impairments; SI, TI A/V Sync
Voice KPIs: MOS; Reliability: call completion rate, dropped call rate; Mean setup time
Diagnostics Logging: Dedicated logging task to gather RF based KPIs from the UE
This live testing program includes state-of-the-art test equipment, trucks, and scheduling systems, walk testing, drive testing, and expert testing engineers who cover a range of geographic locations.
Methodology must account for select devices under test, involve complete operator networks, and provide centralized data collection, analytics, and outputs. To achieve this with comprehensive reliability, a neutral test partner is needed to provide actionable insights from the UE and network sides and with a stable and dependable partner that gives you more than just data. This test partner should be a recognized leader in testing mobile QoE under real-world conditions, with established expertise in 5G validation.
The value of 5G mobile service launch validation
Having a qualified testing solution in place delivers critical information SPs need to maximize the value of their ongoing network investment. Examples of information this kind of validation program would deliver include force-ranked list of network performance by region which helps focus spending on regions where performance is an issue. Comparison of network performance by region versus competition is another category of data which helps a SP’s management focus spending on regions where the competitor is better. As well, evaluation of a service provider’s greatest weakness versus their competitor (e.g., latency), facilitates concentrating budget spend to eliminate competitive disadvantage.
In a more general sense, a service launch validation for 5G networks testing program ensures that deployment strategies and objectives work as planned to realize the best user experience – to gain and retain subscribers. And for the long term, service providers understand the need of continuously strengthening their service infrastructure through ongoing mobile network optimization, fortified by continuous test and assurance of their offerings.
Whether you’re launching 5G, mobile edge computing, new spectrum bands, 5G SA evolution, C-V2X (cellular vehicle-to-everything), Z-axis locations, network slicing, or other 5G mobile services, assessing QoE is critical to success and getting a solid return on your network investment.
To learn more, download our solution brief and read the MEC Latency report: Cutting Through the Edge Computing Hype - 5G MEC Latency Expectations vs. Reality.