WebRTC Trends for 2021 (and beyond)

2021 is set out to be the year of technical debt and quality optimizations. Check out these WebRTC trends to keep up to speed with communication technologies.

Here you can also find the WebRTC predictions and trends 2023 article.

NOTE: There's an update to this one - WebRTC Trends in 2022

Last year was a very interesting and weird year. The vibe of 2020 was dictated by the pandemic and the quarantines around the globe. For those in the communication space, this meant a huge acceleration in demand, scale and the scope of work you had in front of you.

WebRTC and expectations

When I started last year, I talked about the expectations of WebRTC. I tried explaining the concept that WebRTC, more than anything else, is driven by Google and controlled by Google. It was a kind of a follow up to my article on the artificial intelligence roadmap of Google for its “WebRTC Pro” implementation.

Since then, Google introduced noise suppression, background blur and other AI trinkets in Google Meet. All AI features. All were delivered outside of WebRTC but tightly coupled with the WebRTC implementation in Chrome.

What changed since then is the focus. It is great talking about bots and drones. AR, MR and XR. 360 videos, 4K and 8K resolutions. But it gets us nowhere.

We came back to the basics and the basics have changed along with the pandemic.

As developers, we need to follow the trends. Be where our users need us and fill out their requirements. This is also true of WebRTC, and being owned by Google, it means we know where it is (roughly) headed.

Google and WebRTC in 2021

While Google uses WebRTC in multiple services, there are only 2 that matter for WebRTC trends in 2021: Google Meet and Stadia.

Google Meet

In the latest Gartner magic quadrant for meeting solutions (September 2020), here’s who you find:

The leaders? Zoom, Cisco and Microsoft. Google is far behind.

2020 being the year of video meetings, and with Google investing in WebRTC and Meet, this has to hurt.

Google invested heavily in 2020 in and around WebRTC.

You could place their investments in two main areas:

1. Optimizing the code - finally someone took the time to optimize the code and make it more performant and stable on multiple platforms and devices. This is an ongoing work that can still be seen today with each and every release. Google is starting to look at real time video processing as a profession and not a hobby
2. Beefing up the feature set - to meet with what competitors are offering. This trickles back into WebRTC’s capabilities

That trickle-back is important. The 3 leaders in meetings?

• Zoom makes no use of WebRTC, which means it isn’t “limited” by WebRTC’s limitations (or advantages)
• Microsoft Teams offers a subpar experience on browsers. Just try to connect to a video call from Chrome and not the Teams app - you’d be surprised how poor and backward the service feels
• Cisco is improving with WebEx on the desktop. But a lot of the focus and features introduced are outside of the scope of WebRTC. Like the roll out of AV1 support in WebEx

Stadia

Stadia is Google’s cloud gaming platform.

It is still early days for both Stadia and cloud gaming, but a few interesting things have happened in this industry:

• The pandemic got more people to play games. Especially kids. My son plays it now in-between his virtual lessons as well as during the rest of the day. With shelter at home and distancing, this becomes a way to stay connected with friends
• Cyberpunk 2077 video should have been the incentive to join the platform. Gaming consoles like the PlayStation 4 and Xbox One couldn’t handle the game’s high end requirements. Using Stadia or other cloud gaming platform was a reasonable solution. Until bugs were reported about the game itself, causing it to tank globally. Not sure if and how that affects Stadia
• Epic Games battling it out with Apple on its App Store tax rules, with the only potential solution for gaming aggregators being a browser based approach instead of an installable mobile app
• Stadia, being cloud and browser based “enjoys” this

For now, Google seems committed to Stadia. Both Chrome and recently Safari added support to VP9 profile 2. This means a higher color depth than what is common for video conferencing, which is better suited for high end gamers.

Just like Meet, whatever Stadia will need from WebRTC will find its way into WebRTC.

WebRTC Trends in 2021

The trends affecting WebRTC in 2021 are based on two main aspects then:

1. What Google needs for Google Meet and Stadia
2. What many developers are trying to develop with WebRTC

What comes from developers these days is the expansion of remote-everything. There are many domains that aren’t getting heard enough, simply because they are new to the scene. What I think is most interesting is that the mainstream video communications space is still the one setting the agenda for WebRTC.

The 4 biggest trends for WebRTC in 2021 are driven by video communications. Here they are:

Trend #1 - Bigger WebRTC meeting sizes

Our first trend of 2021 for WebRTC? Meeting sizes. Something we've started focusing on only last year.

We used to want higher resolutions. At any given point in time, there was a company pushing the envelope in the resolution for video conferencing. Since we got to HD, that trend stopped. Vendors still tried marketing and selling 4K as a value proposition for video conferencing, but this hasn’t stuck. The high end of the market vanished, leaving us with a new number to focus on. The number of people in a “gallery view”.

With Zoom doing 49, this seems to have become the magic number everyone is aiming towards.

WebRTC was great for smaller meeting sizes, but going beyond 16 video streams in a single session was always challenging. I like using this slide to explain it:

The growing complexity comes with the need to employ ever greater techniques and tricks for optimization. Scaling from 2 users to 10 requires a different approach than scaling towards 50 or 100 users. Aiming for 1,000 users in a meeting needs a slightly different architecture. Going for 20,000 or more necessitates again other tools.

There are now two distinct areas that require large scale WebRTC meeting sizes:

“Traditional” meetings - we had large meetings of 20 or more people, but the people simply convened in 3-4 meeting rooms and connected these meeting rooms. Now each person is a device in the meeting.

Large conferences - we are now trying to copy the real world activity of industry conferences along with entertainment activities (comedians, talk shows, magicians, sporting events, …) and turn them into virtual events. Large online conferences.

These two are different in nature and in the techniques and technical solutions for them.

Google is focused on the “traditional” meetings with their work on Google Meet, which means the optimizations done inside WebRTC’s code as well as enabled on top of it are built to fit this class of problems. The large conferences have a bigger challenge to deal with and less “direct” support from Google and WebRTC.

Trend #2 - De-noising: Background replacement and noise suppression in WebRTC

The second WebRTC trend for 2021 is a bit more surprising. I don't think we would have cared about it much without the pandemic.

Need better media quality? Buy a better camera.

That’s what I did at the beginning of the quarantine. I had to quadruple the number of machines at home with quality peripherals. Instead of only me in meetings we’re now 4 people in meetings, each needing his own different environment. That was obvious to me. Still challenging to do but obvious. We’re also lucky to be able to cater for the four of us in our apartment having a place for each to handle his needs without too much noise seeping out to the others.

Homes with more people? Smaller apartments? How would they handle it?

When we were all in offices things were simpler. The office space was designed (or then redesigned) to meet the needs of video calling. An IT person took care of the space. Someone purchased and installed equipment that fits the needs.

As we’ve all entered a pandemic with quarantine all that careful planning and preparation was thrown out the window. People had to use whatever they had and make do with it. And what did we find out? That there’s background noise and a user’s privacy we need to deal with.

https://www.youtube.com/watch?v=Mh4f9AYRCZY

That child from 2017 who barged into his father’s interview and was live on TV? That’s all of us now. It has become an accepted norm. People working from home. They have a personal life with family and kids, and kids are part of the scenery.

Same for the laundry or other artifacts that now reside behind a person speaking in a video call. How do you make all that go away? How do you reduce the noise of the neighbors running on top of your head while you write these words on a keyboard (literally)?

A rather old/new requirement is to be able to get rid of all of that. Background blurring and replacement. Noise suppression and noise cancellation. All things that were nice to have are becoming common requirements in meeting solutions.

They aren’t part of what comes with WebRTC, but somehow, you need to make them happen with WebRTC.

Trend #3 - A focus on WebRTC user privacy

Zoom and security issues anyone?

I am not here to gloat. Zoom did a bad job at security and privacy before 2020. It did a great job of fixing these issues in record time during 2020.

The issues around Zoom were both about security and privacy. Privacy of the users from other users and hackers, but also from Zoom itself.

This focus on user privacy found its way to WebRTC as well and for the same reason. Zoom is now how every communication company measures itself by, for better or worse.

There are many things to deal with when it comes to WebRTC security and the latest advancement there is E2EE enablement in media servers. The ability to offer end-to-end encryption in a group video call. It is now possible due to the introduction of Insertable Streams to WebRTC.

How is that used? What would it require of you to implement? How would that affect other requirements and features in your service? We are going to find that out during 2021 as more vendors will roll out E2EE solutions with WebRTC.

Trend #4 - WebRTC Investments in VP9 and AV1

Video codec technologies come in stages. The industry at large has started adopting HEVC, with Apple leading the charge. VP9 has been slow to catch up. And we’re already in the next round of codecs with AV1 being hammered as the next big thing and something called VVC breathing down its neck.

WebRTC has been predominantly a VP8 phenomena, with a trickle of H.264. Here’s my estimate on video codecs use in WebRTC:

What is happening now is companies who are looking at VP9 and AV1 trying to make use of them for different use cases and scenarios.

Cisco just announced using AV1 in screen sharing for WebEx in native PC application when that is made possible.

We will see more of that in 2021. Companies experimenting, using and launching products that use more VP9 and even AV1.

An increase in use cases and markets

WebRTC is breaking out to additional markets. Large events, live streaming and even cloud video editing.

All these necessitate new features and capabilities to be added to WebRTC itself.

Now that WebRTC 1.0 is finally being finalized 😀 there is going to be a growing focus by the W3C on what comes next. If you have requirements that require a change in WebRTC, it might make sense for you to join the W3C and make your voice heard in affecting where WebRTC is headed next. Ping me if you’d like to discuss this.

Upcoming WebRTC Trends worksop

Next month I’ll be conducting a workshop that covers these topics. The trends and what to do with them. It will offer actionable advice on what you should do in 2021 and it will be interactive in nature.

My last workshop about differentiation in WebRTC was well attended. Here is what Andrey Abramov of Doxy.me had to say about it:

Thank you very much for the 3 weeks workshop on which you dove us into the WebRTC. It was really interesting and useful. I have learned a lot and look like now I have a better vision of what to do to improve UX of our calls on Doxy.me. Thanks for the records as well! I will be reviewing them from time to time to recall.

It was great! Thank you!

This new workshop, WebRTC trends for 2021, will take place during February, in 3 consecutive sessions of 2 hours each.

Space is limited, so if you are interested, register sooner rather than later.

See you at the workshop.