4K mobile video as a gateway into new services

Ultra-high-definition (UHD or 4K) mobile video is one of the great promises of 5G. However, putting 4K video on a palm-sized smartphone screen can cause readers to wonder why do it at all.

The following looks at the many reasons for 4K.

UHD ‘4K’ video - What is it and what is the technology behind it?

The 16:9 format is actually called UHD-1 (3840 x 2160 pixels) rather than 4K, which is used in digital cinemas with a slightly wider horizontal resolution. The classic Full-HD (1920x 1080 pixels) is now standard and most smartphones can display it. Many data plans now have HD streaming over mobile. 4K video is the next step. It is becoming more common in home television sets and for streaming from video platforms. The visible details are clearly superior but the costs are also much higher as the format uses four times as many pixels as normal full HD. UHD is usually transmitted at double the frame rate of legacy streams, which means the phone needs to process eight times more data. Current smartphones have hardware-accelerated decoding, efficient memory management and can even compress decoded bitmaps moving through the phone video pipeline to the display. Imagine, zipped pictures being internally packed and unpacked using fewer resources than shifting raw pictures in the YUV format.

Transmission has also been changing. Especially when YouTube switched its transport and securing protocols from TCP/TLS to UDP/QUIC, making things leaner and faster but also still reliable. However, the change means defects or missed packets get re-transmitted. The common Full-HD video codec H.264 has been replaced with Google’s VP9, which is also used for 4K video. YouTube is also switching to the new open-source AV1 codec, which is already used for PC based YouTube content and it is only a matter of time before it appears in mobile applications. All codecs are beginning to use the same compression principles, making them more efficient with the same bitrate or providing equivalent quality at lower rates. However, the pure video 4K/60fps bitrate for YouTube is 30Mbit/s with VP9. This rate will fall as 4K becomes more popular on mobile phones and new codecs are installed.

4K video – less a question of smartphone display size and more of how is watched

YouTube ‘knows’ what your device can handle. When requesting a video from YouTube, the default maximum resolution adapts to the smartphone display size. However, smartphones are acting more and more as simple IP access devices that transmit video and other media to connected devices such as monitors or VR glasses. YouTube does more than display videos, it also supports 360° video and virtual reality. Streaming content is mainly determined by the use case and connected devices and less by the display size. Current smartphones can efficiently receive and process 4K video and display it to the user.

New services rise with 5G

Even though 4K video is seen as a key 5G feature, 4K streaming requires gross channel transport capacity of ~50Mbit/s but can also function smoothly under good LTE conditions. Some flagship smartphones come with 4K screens, but seeing more details while simply streaming would require users to hold the device very close and would seem more of an advertisement for technological prowess. We can easily imagine a zoom function for 4K video with no loss of sharpness on a HD screen and enhanced viewing experience for connected monitors or lenses. In general, smartphones will soon focus more on presenting visual content and enabling interactive experience. High resolution video is the gateway for many services such as virtual, augmented reality, extended reality and cloud gaming. Smartphones are transforming into a media router rather than standalone devices.

Efficient 4K video processing is the gateway to many high-resolution visual services, even when only providing two-dimensional pictures or more complicated 3D tracking of motion and reaction to user movements in connected devices.

Many of these new service transport schemes and bitrates are in the current 4K range but interactive applications require faster transport. Well deployed 4G infrastructure can provide the required data rates but 5G is required for truly interactive high-resolution services. Data rate is important but 5G low latency is vital.

YouTube 4K video streaming test on QualiPoc Android

YouTube is no longer merely a steady stream of video files to a phone with a constant resolution and fixed settings. Many variables determine the technical aspects and the perceived quality of received video streams. Apart from switching transport protocols from TCP/TLS to UDP/QUIC, the main change was using MPEG-DASH for variable bitrates. In DASH, video is not transported ‘as one file’ rather than retrieved in short segments of a few seconds. Each segment’s resolution can be requested and the bitrate temporarily lowered or increased based on a device’s channel quality estimate. That reduces freezing up when data-rates are too high but at the expense of details lost due to greater compression. Different video codecs are used in real field applications. There are still VP8 and H.264 in place, some use H.265. VP9 dominates full-HD video today and is used for 4K. AV1 is coming. It is not a matter of one or the other, video streams are encoded with a single one codec that has to be used. Most video platforms will use different encoders for the same video. The client platform chooses the codec best suited to the implemented decoder (which may be hardware accelerated) and the encoded video is then streamed.

Testing video streaming and measuring quality are not just defined by the transport channel for one dedicated video stream. As with test procedures, users request a video as content with an ID or title and not by directly using the URL. Backdoor negotiations between the client player and the video server platform determine the actual stream (codec, protocol and segmentation) and the physical server to deliver the video. The actual video stream and parameters are chosen for the best user experience without any external interaction.

Many variables are in play when requesting a video before it passes the transport channel and airlink to the smartphone. All these settings and the channel itself determine the actual perceived viewer quality. A video streaming test case involves much more than just measuring received bitrates and delay jitter. Perceived quality requires analysis of the video frames actually displayed. Even though Video-MOS is the main quality metric, all the video stream variables are technically just as important. These include bitrate, protocol and codec type reporting along with video resolution over time, frame-rate and time to start playout after video request.

The YouTube video streaming service test from Rohde & Schwarz covers all this information and these measurements. The VMon video analysis algorithm defined in ITU-T J.343.1 is key. No-reference hybrid analysis evaluates and scores video quality with IP-information and real-picture analysis. This algorithm is the most common one used for testing smartphone video quality and is well established in the market. ETSI and ITU recommend using the algorithm for measuring mobile video streaming and it is also used in world-wide network performance scoring methodologies.

With mobile 4K streaming becoming more popular, the VMon algorithm was expanded to analyze and rate video frames at this size and speed. Along with ‘just’ examining larger pictures, the algorithm was adjusted to other 4K aspects with new video codecs to handle 60 frames per second. The brand-new ITU-T P.1204.4 algorithm from Rohde & Schwarz makes efficient and accurate measurement possible. It was standardized by the ITU last year. The algorithm was especially developed and evaluated for videos up to 4K and was also a source for enhancing VMon to 4K video.

In addition to the technical characteristics of a video stream, the method used to view a 4K stream is also important. Connected devices are often used but even viewing of a smartphone display has changed. When phone is used to watch movies by ‘close watching’, users will hold it as close as they do when reading web-pages or even closer. At the same time, smartphone displays are physically bigger, creating a wider viewing angle and improved perception of details. The viewing angle is taken into consideration in each video quality model for predicting perception. This is because the number of visible spatial details perceived by the human retina is an essential factor. The smartphone viewing angle was initially defined using a relative viewing distance of five times display height but as larger phones have come to dominate, the angle was made wider by shortening the distance to four times the display height to emulate closer watching and is seen as a good compromise between smartphones and connected devices. This made it possible to quantify the perceptual improvements of switching from HD to UHD 4K.

The implemented VMon-4K video quality predictor is dedicated to coping with current streaming characteristics, such as newer codecs and higher frame rates while applying a visual model to realistically quantify 4K improvements for smartphone users.

The video quality predictor rates the image quality as seen by the user and also captures, analyzes and scores received and displayed smartphone video images in real-time. The speed optimized algorithm and its excellent performance ensure the real-time analysis of each image even at 4K video rates of 60 frames per second.

The Rohde & Schwarz QualiPoc Androi test suite reports Video-MOS, while also including protocols, codecs, resolution and much more. It helps interpret video streaming by most popular streaming services and optimizes networks for a smooth transmission to truly high resolution video. Together with very short latency, this is the key to providing future video and interactive services.