Mobile network testing blog

Stories & insights

Mobile network testing

Written by Nils Posegga | May 12, 2022

Dimensioning a mobile network benchmarking campaign

The Network Performance Score is used to benchmark networks against competitors over time and even across country boundaries. Rohde & Schwarz implementation in line with ETSI TR 103559 provides the tools needed for data collection and post processing. But what does the dimensioning of a campaign look like when it has representative and statistically reliable results?

Dimensioning a mobile network benchmarking campaign

It starts with the questions:

  • How many measurement systems and teams are needed?
  • How many measurement days should be planned or how many measurement km driven?

Since these questions do not have precise answers, we can provide certain guidelines and methods. The R&S white paper: “How to dimension a mobile network benchmarking campaign“ describes the dimensioning of quarterly benchmark assessments in Germany using a population-centric approach. This entry summarizes the main ideas in the white paper.

Demographic information for Germany is needed as an input – this can be the publicly available population statistics from Wikipedia. The NPS methodology sets minimum sample requirements and selects and weights geographical categories. This allows us to assume that the measurement samples are appropriately distributed. But let us take a step-by-step look:

1. Demographic information

Publicly available information shows the total German population and its distribution among major cities. To cover 25% of the population (a common target for the telecommunications industry), the top 14 cities in Germany are selected and the time and distance needed to reach all these cities are calculated.

Mobile network benchmarking campaign: Route planning
Mobile network benchmarking campaign: Route planning
Rohde & Schwarz drive test system
Rohde & Schwarz drive test system

2. Geographical categories and weighting

When using the NPS methodology suitable geographical categories must be selected that are representative for the target country. The relevance of different categories is defined and weighted for the NPS calculation. In our example we defined the following categories and weighting:

Geographical categories and weighting
Geographical categories and weighting

3. Sample requirements

The NPS methodology requires1000 samples for statistical relevance. This requirement applies to each NPS subset in the final report. We want a statistically relevant result for each of the three categories. We also assume the samples are distributed in line with the category weighting above. This is not a hard NPS requirement but is handy for dimensioning and has the advantage of generating KPI results for the various categories with the same weighting as the NPS score. Please see the Rohde & Schwarz white paper: "Network Performance Score (NPS)” for more information.

4. Calculations

These factors need to be combined and some basic assumptions applied to determine the dimensioning for our benchmarking assessment.

A total distance of ~2300 km is needed to cover the cities. Since other cities and towns will be selected, an estimate is added to determine the total kilometers and hours in the road category and the total number of samples collected.

Duration calculation for a mobile network benchmarking campaign
Duration calculation for a mobile network benchmarking campaign

The duration calculation uses the distance in km and average estimated measurement speed for the category (50km/h).

The total number of samples can be calculated for the road category in line with the initial required sample distribution using the NPS weighting. The estimated samples for the road category represent 15% of all NPS samples. Knowing the total number of samples now allows all the other figures to be calculated along with the category weighting and estimated driving speeds. These can be seen in the table below:

Calculation of the total number of samples
Calculation of the total number of samples

To calculate samples, duration and distance use the following formulas (calculations can work in both directions as needed):

  • duration = #samples / 18 (18 samples per hour based on the call window of 180 seconds and general experience)
  • distance = duration/average duration (per geographical category)

Since the total measurement scope for the remaining city and town categories have been defined and must be distributed among actual cities, the preferred method is to distribute by relative city population. While the list of 14 major cities generally remains fixed, the selection of other cities and towns can vary from quarter to quarter for improved the coverage throughout the year.

Conclusion

Quarterly benchmarking can be done with two measurement teams in ~30days. The assessment delivers a relevant and statistically stable NPS score for Germany and allows for the measurement of quarterly trends.

The full detailed guideline can be found in the Rohde & Schwarz white paper: “How to dimension a mobile network benchmarking campaign”.

Related stories

Impressions from managed services team benchmarking Africa

Read more

Network Performance Score: How to conduct NPS benchmarking campaigns

Read more

Network Performance Score: Configurator for step-by-step campaign setup

Read more

Network Performance Score: Templates for easy support in all products

Read more

Subscribe MNT blog

Sign up for our newsletter

Stay up to date and get stories and insights with our frequent mobile network testing newsletter.

Stories by category

Benchmarking & optimization

More information

Field services & interference hunting

More information

Innovations in mobile network testing

More information

Testing from RF to QoE

More information

Request information

Do you have questions or need additional information? Simply fill out this form and we will get right back to you.

Marketing permission

Your request has been sent successfully. We will contact you shortly.
An error is occurred, please try it again later.