Mobile network testing blog

Stories & insights

Mobile network testing

Written by Jens Berger | November 22, 2021

Procedures for mobile network testing and efficient data collection

How is network performance tested and rated? The answer to this question is essential to mobile network operators, mobile infrastructure vendors, regulators and authorities responsible for ensuring telecommunication services meet performance standards. Mobile phone users are also very interested in having a true and realistic view to what their providers can deliver.

Procedures for mobile network testing and efficient data collection

It's hard to imagine our world without mobile communications and smartphones are much more than just telephones. They are the primary access to internet services for many. When talking about network performance, analyses should include all the typical use cases for the most benefit when optimizing the network experience.

Network performance is more than just data rate testing, which is important. Networks can react differently when faced with more complex demands for data transmission and when connected to content servers. Benchmarking, drive tests and walk tests typically involve applications such as video streaming, messaging or web-browsing for an end-to-end view of the performance.

End-to-end tests of popular services are called ‘app tests’. However, they do not test the app but the service behind it. The main issue is how the network performs relative to the individual app requirements.

Test case classification

Classifying test cases is a useful first step. This should not be done according to app or brand name but by the needs served and data transmission requirements. OTT messaging applications also offer telephony or data transfer and social media platforms provide streaming, telephony or push-to-talk services along with the posting of pictures.

When assigning a test case to a class, it should be grouped by the service provided and network requirements.

Figure 1: Test case classification
Figure 1: Test case classification
Open Lightbox

If keeping with such classification, testing WhatsApp® telephony is completely different in terms of network usage and has completely different KPIs than WhatsApp® messaging, Facebook® Post Picture and video streaming in Facebook® Watch. They may share a brand name but the individual services are for different user needs, present different challenges to networks and result in different KPIs to quantify performance.

This would mean that YouTube® video streaming and Facebook® Watch should be grouped together, just as WhatsApp® messaging and OTT messengers or WhatsApp® telephony should be matched with other telephony services and even VoLTE. The same set of KPIs apply to similar applications, such as ‘call setup time’ or ‘speech quality’ for telephony or ‘time to post a picture’ for messenger services or social media portals. Using the same KPIs allows a direct comparison for individual services and their end-to-end performance.

Rohde & Schwarz mobile network testing organizes solutions by grouping tests by subscriber services and network resources use, from test definition and KPI alignment to reporting and documentation. This keeps those things together that belong together.

Figure 2: Rohde & Schwarz mobile network testing organizes solutions by grouping tests by subscriber services and network resources use, from test definition and KPI alignment to reporting and documentation
Figure 2: Rohde & Schwarz mobile network testing organizes solutions by grouping tests by subscriber services and network resources use, from test definition and KPI alignment to reporting and documentation
Open Lightbox

Efficient data collection for mobile network performance measurement

Data collection should be efficient. Equipment and resources are expensive but produce high measurement resolution for time and physical location using moving drive or walk tests. The sparser the results, the more time and space gaps that go unobserved.

Efficient data collection involves gathering as much network insight as possible in a given period of time. This does not always mean transmitting as much data as possible in a short time as with speed tests over the network or covering as many apps as possible. The task is obtaining as many different transmissions as possible for a given period of time, so that performance can be specified for the entire network or a portion of it.

Typically, the focus is more on network performance and less on comparing the same type of different services. Applications that have identical or similar network resource use and are in the same group generate similar data patterns, place the same demand on networks and have similar KPIs. These services may use different compression schemes, time-outs or side-information and temporary bad connectivity is also possible. However, the basic data transfer scheme transfer and load patterns are the same. Therefore, running different applications from the same group provides almost no additional insight from the network perspective. If a network smoothly delivers YouTube® video streams, probably it can do the same for other video streaming services. Also, if WhatsApp® messaging works well, other OTT messenger apps will not be treated substantially differently during transmission.

Data speed tests are not enough on their own, they only provide information about the network under full load in a single transmission scheme. The network may behave differently when less data is transmitted, when transport is bulk-wise or other protocols are used.

When focusing on overall network performance, different applications should be combined in a test cycle, placing different demands to the network and resulting in the varied data patterns and protocols of a typical user. In line with the classification above, the tested services should represent the individual application and service classes based on their use of network resources. This should include popular services such as the retrieving of web-content, streaming media and posting information along with simple data downloads and uploads and telephony.

Integrative performance testing

ITU-T E.804.1, ETSI TR 103 559 and TR 103 702 also reference this spread across many services. The scoring methodology in ETSI TR 103 559 specifically compiles points for individual service classes and their dedicated KPIs and reports overall network performance that includes different network usage aspects split by region. The following scheme shows the aggregation methodology. The example uses ‘highways’ as regions, but the scheme is identical for all other regions in the campaign compiled to form an overall network score.

Figure 3: Integrative performance testing
Figure 3: Integrative performance testing
Open Lightbox

Note: Rohde & Schwarz supports the Network Performance Score methodology in all their mobile network testing products and solutions, pre-configured measurement campaigns and dedicated views in post-processing tools.

Returning to the individual service classes such as web-content delivery or streaming, ETSI TR 103 559 provides reasonable examples of the importance of individual service and app classes. Originally designed to generate an integrative network performance scores, it can also be seen as a guideline for optimizing tasks to provide the best network performance for daily smartphone use.

Figure 4: Typical weight for individual data services in performance scoring
Figure 4: Typical weight for individual data services in performance scoring
Open Lightbox

The example is clearly weighted to the delivery of web-content as files (web-pages, pictures, maps, …) or audio and video media streams. It is a rough estimation of the current situation.

Even though new use cases will be created under 5G, the picture may not change much but the services behind each component will become more interactive. Cloud gaming is a good example, a media stream is still delivered but it reacts to a player’s actions. In the same vein, interactive social media platforms will arise and deliver reactive web content.

Latency and interactivity will become just as important as data speed and transfer completeness are today. Latency and interactive applications are already at the heart of ETSI TR 702 103: QoS parameters and test scenarios for assessing network capabilities in 5G performance measurements.

The product line at Rohde & Schwarz mobile network testing already includes realistic transport latency and network interactivity measurements in their Interactivity Test.

Related stories

Interactivity test: Distance to server impact on latency and jitter

September 28, 2021 | by Jens Berger

Data transport latency is a very – if not the most – important factor for interactive and real-time use cases in future mobile networks. Today the focus is often on maximizing bitrate and transport capacity for fast downloading, but future applications will require extremely low latency for real-time interactions.

Read more

4K mobile video as a gateway into new services

May 27, 2021 | by Jens Berger

Rohde & Schwarz MNT has implemented YouTube 4K video testing for new video services. Networks are ready to transmit 4K video, smartphones can process it and video apps are streaming it. The next logical step is to test mobile 4K video streaming from YouTube, the internet’s most popular mobile video streaming service.

Read more

End-user quality of experience (QoE) 5G network evaluation for three major US network operators

March 31, 2021 | by Jens Berger

The independent wireless industry research experts at Signals Research Group (SRG) worked with Rohde & Schwarz MNT on a 5G network quality benchmarking study in the Dallas area, a major U.S. telecommunications hub, comparing the performance of three tier-1 US network operators.

Read more

Subscribe MNT blog

Sign up for our newsletter

Stay up to date and get stories and insights with our frequent mobile network testing newsletter.

Stories by category

Benchmarking & optimization

More information

Field services & interference hunting

More information

Innovations in mobile network testing

More information

Testing from RF to QoE

More information

Request information

Do you have questions or need additional information? Simply fill out this form and we will get right back to you.

I want to receive information from Rohde & Schwarz via

Marketing permission

What does this mean in detail?

I agree that Rohde & Schwarz GmbH & Co. KG and the Rohde & Schwarz entity or subsidiary company mentioned in the imprint of this website, may contact me via the chosen channel (email or postal mail) for marketing and advertising purposes (e.g. information on special offers and discount promotions) related to, but not limited to, products and solutions in the fields of test and measurement, secure communications, monitoring and network testing, broadcast and media, and cybersecurity.

Your rights

This declaration of consent may be withdrawn at any time by sending an email with the subject "Unsubscribe" to news@rohde-schwarz.com. Additionally, a link to unsubscribe from future email advertisements is contained in each email sent. Further details on the use of personal data and the withdrawal procedure are set out in the Statement of Privacy.

Your request has been sent successfully. We will contact you shortly.
An error is occurred, please try it again later.