FEC (Forward Error Correction) consist on include redundant information about a RTP packet in the following packets. That way if one of the packets is lost the data can still be recovered from the information received in the next packets without requiring retransmissions. That makes it specially suitable for networks with high latency or real time requirements where retransmissions are less of an option.

Nowadays there are different implementations of Forward Error Correction inside WebRTC clients including Opus FEC, ULPFEC and FlexFEC (disabled behind a flag). These schemas are negotiated in the SDP Offer/Answer exchange.

ULPFEC is defined in RFC 5109 and uses XOR across multiple packets to generate this redundant information and be able to recover lost packets in the receiver side if needed. ULPFEC has the ability to provide different levels of protection for different packets by selecting the amount of bytes being protected and the number of previous packets the XOR is applied to.

In case of WebRTC, ULPFEC information is transmitted in the RTP Packets using the RED format.

The post ULPFEC (Uneven Level Protection Forward Error Correction) appeared first on BlogGeek.me.

The primary motivation of sending redundant data is to be able to recover packets lost under lossy network conditions. If a packet is lost then the missing information may be reconstructed at the receiver from the redundant data that arrives in the following packet(s).

The use of RED is negotiated in the SDP as an additional payload type and when used the audio/video RTP packets are packaged using RED format with a 6 bytes header, before the primary and secondary payloads conveying the actual audio/video packet and the redundant information.

An example of a redundant information schema is ULPFEC.

See also FEC.

The Role of RED in WebRTC

In the context of WebRTC, RED can be used to increase audio resiliency to packet loss. For many years, WebRTC relied on the internal forward error correction built into Opus to supply its packet loss resiliency. While the results were fine, they weren’t great. To improve on this, developers have sought out to use RED as a secondary mechanism, beefing up resiliency.

There are two opposing opinions here:

1. FEC needs to be done in an intelligent way, using as little extra bitrate as possible
2. “Brute force” solutions such as RED are useful even of expensive in bitrate

Due to this, WebRTC doesn’t really support RED “out of the box”, but rather enables implementers to add it through the use of media servers and Insertable Streams. Those adopting RED do so with an understanding that it doubles or triples the bitrate the audio bitrate – which is still quite low compared to the bitrate requirements of the video stream in a video call.

The post RED (REDundant coding) appeared first on BlogGeek.me.

A receiving end can send a FIR message to the sender to indicate that it needs to refresh its received video.

Video is usually sent not as a compressed set of frames, but rather a compressed set of delta frames, which rely on previous frames to be decoded and displayed. If some of the packets get lost along the way, then it becomes somewhere between hard to impossible to continue decoding the stream without some remedy.

Sending out a FIR message indicates the sender to “reset” his encoder and start with a fresh frame.

The post FIR (Full Intra Request) appeared first on BlogGeek.me.

A PLI can be sent when the receiver of the media lost a full frame or more. It is sent over RTCP.

A receiver of this message can decide to resent a full frame if he has one available in cache and finds it usable.

This message is similar to a FIR message but a bit more lenient in its heuristic.

Importance of PLI in WebRTC

In a WebRTC video call, maintaining the quality of the video stream is paramount. Networks can be unpredictable, and data packets, including video frames, can get lost or arrive out of order.

When a receiver detects that a picture is missing or corrupted, it sends a PLI message back to the sender. The sender can now ensure that the video stream remains intact and of high quality by re-transmitting the lost picture or sending a brand new intra-frame to reset the video feed.

PLI plays a crucial role in the error recovery process within a WebRTC video communication setup.

By signaling the sender about the loss of picture data, it helps in triggering the necessary corrective actions to restore the video stream’s quality.

Implementation of PLI

The implementation of PLI in a WebRTC setup involves both the sender and the receiver. The receiver monitors the incoming video stream for any missing or corrupted pictures.

When such an event is detected, it generates a PLI message and sends it back to the sender via RTCP. The sender, upon receiving the PLI message, can re-transmit the lost or corrupted picture to the receiver or it may reset the stream using an i-frame. It may also decide to ignore the message altogether.

The PLI mechanism is a part of the error resilience features provided by WebRTC to handle adverse network conditions and ensure a smooth and high-quality video communication experience.

The post PLI (Picture Loss Indication) appeared first on BlogGeek.me.

NACK is a way for the receiving end to indicate it hasn’t received a specific packet.

A NACK message is sent over RTCP to the sender of the media, which in turn needs to decide if it will retransmit the lost packet based on its availability in its cache and its estimate to the usefulness of the retransmission (will it be possible to use it once received).

Mechanism of NACK

The mechanism of NACK is straightforward yet crucial for maintaining the quality of the communication link. When a receiver detects the absence of a specific packet, it triggers a NACK message.

This message is then transmitted over RTCP to the sender of the media. The responsibility now lies on the sender to evaluate the request for retransmission.

Decision on Retransmission

Upon receiving a NACK message, the sender has to make a calculated decision regarding the retransmission of the lost packet.

This decision hinges on two primary factors:

1. The availability of the packet in its cache
2. The estimated usefulness of the retransmission

The sender evaluates whether the retransmitted packet, once received, will still hold value for the ongoing communication. If it doesn’t, he won’t retransmit it.

Importance in WebRTC

The implementation of NACK in WebRTC directly impacts the quality and reliability of the real-time communication. It is focused on video streams and a lot less on audio streams.

By providing a mechanism to request for lost packet retransmission, NACK plays a vital role in mitigating the effects of packet loss, thereby enhancing the overall user experience in a WebRTC environment.

The post NACK (Negative Acknowledgement) appeared first on BlogGeek.me.

When a stream of packets is received by the media engine, it is not guaranteed that everything sent gets received.

In case not all packets are received, and no FEC is employed (or is usable), packet loss concealment comes into play.

Packet loss concealment isn’t standardized anywhere and is left for the implementer of the media engine and codec.

The post PLC (Packet Loss Concealment) appeared first on BlogGeek.me.

WebRTC uses IP networks to transport its data. IP networks are prone to errors causing packet loss. This means that media data sent by one WebRTC client to another might not be received on the other end due to congestion or network corruption.

FEC is a mechanism in which media packets are duplicated and sent multiple times across the network without any delay. This means that even if some of the packets aren’t received, the media stream can still be parsed and decoded properly. This assumes that the packets that got lost during transit have been sent multiple times.

FEC exist in WebRTC in different ways:

• As part of the implementation of the Opus voice codec
• REDundant coding for audio, where each audio frame can be sent more than once

The post FEC (Forward Error Correction) appeared first on BlogGeek.me.