In the past few years, the video streaming industry has seen immense interest in low-latency streaming protocols. The majority of interest is on delivering videos with a sub-five-second delay, making them comparable with delays in live broadcast TV systems. Attaining such low delay is critical for streaming live sports, gaming, online learning, interactive video applications, and others.
Developing the Technology for Low-Latency Streaming
The delays in conventional live OTT streaming technologies such as HLS
and DASH
are much longer. They’re caused by relatively long segments (4-10 seconds) and a segment-based delivery model, requiring complete delivery of each media segment before playback. Combined with the buffering strategies used by the HLS or DASH streaming clients, this typically produces delays of 10-30 seconds, or even longer.
To combat delays, recent evolutions of HLS and DASH standards, known as Low-Latency HLS (LL-HLS)
and Low-Latency DASH (LL-DASH)
, have introduced two key tools:
Chunked video encoding. This is an encoding strategy that produces video segments structured as sequences of much shorter sub-segments or chunks.
Chunked segment transfer. This is an HTTP transfer mode that enables transmission of shorter video chunks to streaming clients as soon as they are generated.
A streaming client can join a low-latency live stream from any Stream Access Point (SAP) which is made available by the live transcoder on segment boundaries or chunk boundaries. Once joined, a player would only need to buffer the latest video chunk generated by the encoder before decoding and rendering it. Considering that each segment can be split into several chunks (typically 4-10), this reduces the delay significantly.
Several open-source and proprietary implementations of low-latency servers and players are currently available on the market, including one by Brightcove. Many of them have demonstrated lower streaming delay when only a single-bitrate stream is used and when they stream over high-speed network connections. However, their performance under more complex and realistic deployment environments has not been well studied.
Testing the Performance of Low-Latency Streaming
In our paper published in ACM Mile-High-Video 2023 (link forthcoming) as well as a presentation at Facebook@Scale
, we reported some results after testing the performance of low-latency systems.
First, we developed an evaluation testbed for both LL-HLS and LL-DASH systems. We then evaluated various low-latency players using this testbed, including Dash.js, HLS.js, Shaka player, and Theo player. We also evaluated several of the latest bitrate adaptation algorithms with optimizations for low-latency live players. The evaluation was based on a series of live streaming experiments. These experiments were repeated using identical video content, encoding profiles, and network conditions (emulated by using traces of real-world networks).
Table 1 shows our live video encoding profiles for generating the multi-bitrates, low-latency DASH/HLS stream.
.tg {border-collapse:collapse;border-spacing:0;}
.tg td{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px;
overflow:hidden;padding:10px 5px;word-break:normal;}
.tg th{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px;
font-weight:normal;overflow:hidden;padding:10px 5px;word-break:normal;}
.tg .tg-baqh{text-align:center;vertical-align:top}
.tg .tg-amwm{font-weight:bold;text-align:center;vertical-align:top}
| Rendition | Video resolution (pixels) | Video codec and profile | Bitrate (kbps) |
|---|---|---|---|
| low | 768 x 432 | H.264 main | 949 |
| mid | 1024 x 576 | H.264 main | 1854 |
| high | 1600 x 900 | H.264 main | 3624 |
| top | 1920 x 1080 | H.264 main | 5166 |
Table 1. Live video encoding profiles
Figure 1 shows the workflows of our low-latency DASH and HLS streaming testbeds.
Figure 1. System setup used for evaluation of low-latency players.
Figure 2 shows the available network bandwidth of our emulated LTE mobile network. The download bandwidth of the low-latency players is controlled by our network emulation tool and the network traces.
Figure 2. Available bandwidth of emulated LTE mobile networks (Verizon and T-Mobile)
A variety of system performance metrics (average stream bitrate, amount of downloaded media data, streaming latency) as well as buffering and stream switching statistics were captured and reported in our experiments.
These results have been subsequently used to describe the observed differences in the performance of LL-HLS and LL-DASH players and systems.
A few plots from our study can be seen in the figures below.
Figure 3. Dynamics of bitrate switches reported by LL-HLS and LL-DASH players.
Figure 4. Comparison of variations of latency reported by LL-HLS and LL-DASH players.
Performance statistics – T-Mobile LTE network
.tg {border-collapse:collapse;border-spacing:0;}
.tg td{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px;
overflow:hidden;padding:10px 5px;word-break:normal;}
.tg th{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px;
font-weight:normal;overflow:hidden;padding:10px 5px;word-break:normal;}
.tg .tg-1wig{font-weight:bold;text-align:left;vertical-align:top}
.tg .tg-baqh{text-align:center;vertical-align:top}
.tg .tg-amwm{font-weight:bold;text-align:center;vertical-align:top}
.tg .tg-0lax{text-align:left;vertical-align:top}
| Player/Algorithm | Avg. bitrate [kbps] | Avg. height [pixels] | Avg. latency [secs] | Latency var. [secs] |
Speed var. [%] | Number of switches | Buffer events | Buffer ratio [%] | MBs loaded | Objects loaded |
|---|---|---|---|---|---|---|---|---|---|---|
| DASH.js default | 2770 | 726 | 3.06 | 0.21 | 10.4 | 93 | 38 | 7.99 | 352.2 | 256 |
| DASH.js LolP | 3496 | 853 | 5.65 | 4.59 | 22.7 | 70 | 53 | 21.96 | 369.4 | 210 |
| DASH.js L2all | 3699 | 908 | 4.14 | 3.18 | 19.9 | 5 | 19 | 7.99 | 368 | 147 |
| Shaka player (dash) | 3818 | 916 | 4.92 | 2.06 | 0 | 16 | 5 | 4.66 | 360.3 | 155 |
| THEO player (dash) | 4594 | 993 | 6.16 | 0.01 | 0 | 27 | 0 | 0 | 418.7 | 152 |
| HLS.js default 2020 | 1763 | 562 | 10.08 | 10.91 | 8.1 | 26 | 2 | 9.8 | 130.7 | 589 |
| HLS.js LolP 2020 | 1756 | 560 | 5.97 | 0.2 | 6.1 | 24 | 0 | 0 | 148.1 | 688 |
| HLS.js L2all 2020 | 1752 | 560 | 6 | 0.23 | 5.9 | 34 | 0 | 0 | 133.1 | 686 |
| HLS.js default 2023 | 3971 | 895 | 8.93 | 1.13 | 0 | 8 | 0 | 0 | 360.8 | 613 |
| Shaka player (HLS) | 3955 | 908 | 7.18 | 2.23 | 0 | 14 | 7 | 3.8 | 230 | 475 |
Table 2. Performance statistics – T-Mobile LTE network
Evaluating the Performance of Low-Latency Streaming
While LL-HLS and LL-DASH worked well in unconstrained network environments, they struggled in low bandwidth or highly variable networks (which are typical in mobile deployments). Observed effects included highly variable delays, inability to prevent buffering, and frequent bandwidth switches or inability to use the available network bandwidth. Some players simply switched into non-low-latency streaming under such challenging network conditions.
As promising as LL-HLS or LL-DASH are in theory, there is still some room for these technologies to mature.
At Brightcove, we are continuing to work on best-in-class implementations of algorithms for low-latency streaming clients as well as encoder and server-side optimizations. We intend to make our support for low-latency streaming highly scalable, reliable, and fully ready for prime time.
This blog was originally written by Yuriy Reznik in 2021 and has been updated for accuracy and comprehensiveness.