Mobile network testing blog

Stories & insights

Mobile network testing

Written by Meik Kottkamp | September 1, 2023

Interactivity Test: Testing service availability (part 9)

5G NR is a cellular technology that supports ultra reliable low latency (URLLC) services. Latency and high throughput have become important KPIs and the interactivity test from Rohde & Schwarz can take measurements in the field by combining access to transport capacity (bit rate), transport latency and transport continuity, all in a single test case.

Interactivity test: Testing service availability (part 9)

In the round trip time (RTT) measurements, a mobile device initiates the process by sending a signal towards a server connected to the cellular network and then back to the device. Industrial applications that use private 5G NR networks require both low latency and high service stability (high connection availability and reliability). Requirements vary and range from 99.99% to 99.9999% service availability. Note that 99.9999% service availability equals 31 seconds of downtime per year and raises the question as to whether testing can verify such requirements. This article explores the interactivity test as a possible way to meet such stringent testing requirements.

Test method and solution to determine service availability

The test solution uses the commercially available interactivity test with 5G STS or Qualipoc Android. The setup below illustrates the measurement in a 5G smart factory. RTT measurements with interactivity test are initiated by the user device, e.g. a 5G module connected to the 5G STS:

Figure 1: Measurement in a 5G smart factory

The interactivity test comes with a wide range of predefined traffic patterns. Note that the TWAMP-based solution can vary packet size, packet frequency and delay budget. The delay budget is the acceptable delay for data packets, those arriving outside the time in the budget are labeled “discarded” in the analysis. Details for certain traffic patterns can be seen in the following.

Table 1: Data profile characteristics

Method for determining service availability
Certain use cases can have very high service availability requirements. The table below provides concrete examples from 5G-ACIA, ZVEI or ABI research.

Table 2: Concrete examples from 5G-ACIA, ZVEI or ABI research.

Note that a 99.9999% requirement equals 31 seconds of downtime a year, raising the question of whether such stringent requirements can be tested in a reasonable amount of time. Fortunately, interactivity tests send out an enormous number of packets at very short intervals (0.77 ms to 16.1 ms) so that even stringent requirements can be verified in line with the calculations below.
Multiple sessions (M) can be run in sequence while multiple packets (N) are transmitted during one session within the observation period.

Figure 2: Multiple sessions (M) can be run in sequence while multiple packets (N) are transmitted during one session within the observation period

Different applications tolerate a different number of packets, which are lost or arrive too late. In our analysis we distinguish four cases. If 99.99% service availability is required (SAreq), a service will fail because a (1) single packet is lost or fails to arrive in time (2) two consecutive packets are lost or delayed or (3) four consecutive packets are either lost or arrive later than specified in the time budget. Error cases (E) simply describe the case, if the latency for each packet is too high relative to the specified delay limit based on whether just a single packet or multiple packet delays can be tolerated. The following formula determines whether the service availability requirement has been met:
Accordingly, a specific number of error cases (E) will lead to a service availability rate (SA) as per

Service availability rate (SA)

Results

We tested a private 5G NR SA network at our plant in Teisnach, Germany. Rohde & Schwarz operates this private 5G SA network in the n78 spectrum band with 80 MHz bandwidth (3.72 to 3.8 GHz). We applied the long traffic pattern to our measurements to transmit/receive 4500 packets within 30 seconds. We further configured our test so that a single test was run every 10 minutes over a total period of 24 hours. 607500 packets were sent during our one-day observation period.

The following picture illustrates the measurement results taken in mid-April 2023. First note the very stable performance over the day even though the latency values are sometimes quite high.

Figure 3: Measurement results taken in mid-April 2023

Let’s take a closer look at the absolute values using the method for determining service availability described in section 2. The median RTT for the day is 16.39 ms. Note that latency optimizations were not applied to the 5G NR SA network and none of the 3GPP Rel16 features, which are specifically designed to support URLLC, were implemented yet. The device under test was a commercial high-end smartphone running the Qualipoc Android interactivity test solution. Depending on the requirements, we assumed 40ms RTT for moderate use cases with a tolerance level at the application for service availability of better than 99.99%.

Table 3: The absolute values using the method for determining service availability described in section 2

To verify 99.9999% service availability and see at least 10 error cases (single packet outside the delay budget) during measurement, a total of 10 million packets would need to be sent. Using the long traffic pattern as a baseline (4500 packets in 30 seconds), total testing time would exceed 18.5 hours per day without any pause between sessions. Note that multiple parameters influence required overall test time, such as the traffic pattern (how many packets are sent over a time period), the application of interest (how many consecutive packets may be lost/delayed) and statistical certainty (the minimum number of errors to be measured). However, reviewing the sample calculations in this section, we find that even tough requirements only need days of testing.

Conclusions

5G NR performance measurements remain essential when setting up and operating commercial networks, specifically when private/local networks must meet very low latency requirements. The interactivity test combines testing round-trip latency, packet delay variation and packet error rate in a single test. The inbuilt flexibility for various traffic patterns lets the solution verify service availability in a relatively short amount of time, even when very stringent service availability requirements apply.

Read more about interactivity test in part 1 - 8 of this series and download our white paper "Interactivity Test"

Interactivity test: QoS/QoE measurements in 5G (part 1)

Interactivity test: Concept and KPIs (part 2)

Interactivity test: Examples from real 5G networks (part 3)

Interactivity test: Distance to server impact on latency and jitter (part 4)

Interactivity test: Impact of changing network conditions on latency and jitter (part 5)

Interactivity test: Packet delay variation and packet loss (part 6)

Interactivity test: Dependency of packet latency on data rate (part 7)

Interactivity test: ITU-T Recommendation G. 1051 (part 8)

White paper "Interactivity test"

Related stories

Interactivity Test: ITU-T Recommendation G.1051 (part 8)

Read more

Interactivity test: Dependency of packet latency on data rate (part 7)

Read more

Interactivity test: Packet delay variation and packet loss (part 6)

Read more

5G NR and LTE latency analysis in a public network (part 1)

Read more

Subscribe MNT blog

Sign up for our newsletter

Stay up to date and get stories and insights with our frequent mobile network testing newsletter.

Stories by category

Benchmarking & optimization

More information

Field services & interference hunting

More information

Innovations in mobile network testing

More information

Testing from RF to QoE

More information

Request information

Do you have questions or need additional information? Simply fill out this form and we will get right back to you.

Marketing permission

Your request has been sent successfully. We will contact you shortly.
An error is occurred, please try it again later.