What are the Implications of VP9 (or H.265) on WebRTC?

What happens when we switch from VP8 to VP9 codec?

We have an MTI decision for WebRTC's video codec. But our future is going to be moving rather rapidly towards VP9 (or H.265 for that matter) - there are early indications of VP9 availability in Chrome Canary (=beta) under a command line flag.

What happens then? Why will this shift occur?

Putting all business related issues aside (=patents), video codecs differ from each other in three main parameters:

1. Compression efficiency
2. Complexity
3. Acceleration availability

The rest are just details.

Let's look what happens when we switch the lights on and migrate to VP9.

1. Compression efficiency

Each new generation of codec is widely thought to be twice as efficient in compression. What does this mean?

• For the same bitrate you can get a lot better video quality; or
• For the same video quality you need to sacrifice half the bits you had to before

The improved compression efficiency of VP9 over VP8/H.264 means we can improve video quality and increase resolutions or we can decide to reduce the bandwidth we need for a given call.

In our Retina resolution world, higher resolutions means better video quality. We won't be headed here in 2015. Wait until 2017-2018 to see 1080p and 4K resolutions starting to creep into mass market consumer services with WebRTC.

On the other hand, using half the bandwidth is something that VP9 will be used for immediately:

• If you run a service, then TURN relays will cost half their price for the same number of sessions. SFUs and other server side media processing entities will incur less bandwidth costs as well.
• If you target markets with no broadband, or lower uplink speeds, then VP9 is great to push more quality in the same pipe

2. Complexity

Better performance comes at a price. In our case, 2-3 times the CPU power is required to compress using VP9 than it is using VP8 or H.264.

This means battery life on smartphones will be a challenge for VP9. Desktops are going to eat up their CPUs to crunch VP9 videos.

TURN servers won't be affected from this added complexity, as they don't deal with the media coding itself. If you need transcoding or any other server side media processing on the video plane, then you are going to pay in CPU. You will need more servers to crunch VP9 data.

3. Acceleration availability

Video codecs live through acceleration. You can use software codec implementations for them, but these eat up CPU and affect user experience (and/or battery life). This is why for any serious mass market implementation you will be needing to rely on hardware acceleration as much as possible.

VP9 is too young to have hardware acceleration to speak about - especially when it comes to mobile. It will take an additional 18 months in the very least until we start seeing anything that fits. Think Nexus 8 or something similar.

Until then, we live in a software compression world for VP9.

Why is this important?

The fight between VP8 and H.264 was an important one, but one that will be buried in the pages of history come 2016. Our WebRTC services will probably continue to use these two codecs until 2020 or something similar. A lot before that, VP9 and maybe H.265 will be introduced into WebRTC. These will bring with them new challenges but huge opportunities in improved video quality.

WebRTC with VP9 will bring video quality to the browser at a level that today's video conferencing systems can't. With the browser's ability to update at will in a 6-weeks release cycle, there is nothing to stop the browser from becoming the best real time communications client in 2015.

If you are investing in video coding technologies for your WebRTC service, you should seriously start thinking of your roadmap towards VP9 - at least if staying relevant is one of your aspirations.

We have to note that the switch to VP9 never really took place. At list not en masse. It now seems like the real battleground is going to be using AV1 vs HEVC in WebRTC. I wonder which video codec will be adopted by developers.