Hardware Acceleration is Real

Technology
June 20, 2016

Hardware acceleration for video codecs is almost mandatory - vp9 codec is getting a performance boost.

VP9 codec is getting a performance boost

There are three things that keep VP8 in the game when compared to H.264:

  1. It was the only video codec in Chrome for WebRTC in the last 5 years, giving it a headstart in deployments
  2. H.264 while available in mobile chipsets isn't always accessible for the developer (or works as it should when it is accessible)
  3. VP8 and H.264 are rather old now, so software implementations of them are quite decent

With VP9 codec, the main worry was that it will be left behind and not get the love and attention from chipset vendors - leading it to the same fate as VP8 - abysmal, if any, hardware acceleration support. It is probably why Google went to great lengths to make it running on YouTube so soon and is publicizing its stats all the time.

👉 Check out if VP9 codec is suitable for your WebRTC application

This worry is now rather behind us. Recent signs show some serious adoption from the companies that we should really care about:

#1 - ARM and VP9 acceleration

Mobile=ARM

Without checking stats, I'd say that 99% or more of all smartphones sold in the past 5 years are based on ARM.

If and when ARM decides to support a feature directly, that brings said feature very close towards world domination in future smartpones.

Which is somewhat what happened last week - ARM announced its Mali Egil Video Processor with VP9 acceleration.

Here's a deck they shared:

http://www.slideshare.net/PhilHughes14/arm-mali-egil-technical-preview
VP9 Codec slide share by Roger Barker

Being farther away from chipsets than I were 5 years ago, it is hard for me to say if this is an integral part of an ARM processor, but I believe that it isn't. It is an add-on component that takes care of video processing that chipset vendors add next to their ARM core. They can source the design from ARM or other suppliers - or they can develop their own.

Not sure how popular the ARM alternative is for video processing, but they have the advantage of being the first alternative for any chipset vendor (hell - they already source the ARM core itself, so why not bundle?). Which also means every other vendor needs to match up to their feature set - and improve on it.

Now that VP9 encode/decode capabilities are front and center in the ARM Mali Egil, it has become a mandatory checkmark for everyone else as well.

#2 - Intel on the video codec

If ARM is the king of mobile, then Intel rules the desktop.

As with ARM, I haven't been following up on Intel CPU acceleration lately. And as with ARM, it was Fippo who got my attention with this link here: the new Intel Media SDK.

For those who don't know, Intel is providing several interesting software packages that make direct use of its chipset capabilities. Especially when it comes to optimizing different types of workloads. The Intel IPP and Media SDKs handle media related processing, and are quite popular by low level developers who need access to such facilities.

From the release page itself:

With this release we are happy to announce new full hardware accelerated support for HEVC and VP9.

  • So... HEVC (=H.265) has encode and decode while VP9 only has decode support.
  • Probably because HEVC has been in the works for a lot longer than VP9, but there's hope still.

#3 - Alliance of Open Media

The Alliance of Open Media. I've published a recent update on the alliance.

Intel was there from the start. The recent additions include ARM, AMD and NVIDIA. Sadly, Qualcomm isn't part of the alliance.

I am sure additional chipset vendors will be joining in the coming months - there seems to be a ramp up in memberships there.

The alliance is working on AV1, a video codec that is planned to inherit VP9 and have wide industry acceptance and adoption. We are already seeing discussions around HEVC vs AV1 in WebRTC.

While the alliance is about what comes after VP9, it is easy to see how these vendors may sway to using VP9 in the interim.

The Future - VP9 codec and beyond

The future is most definitely one of royalty free video codecs. We've got there with voice, now that we have OPUS (though Speex and SILK were there before to pave the way). We will get there with video as well.

Coding technologies need to be accessible and available to everyone - freely - if we are to achieve Benedict Evans' latest claims: Video is the new HTML. But for that, I'll need another post.

So… which of these video codecs should you use in your application? Here’s a free mini video course to help you decide.

FAQ on video codec acceleration

✅ Can VP9 work without hardware acceleration?

Yes it can, and it does.
Google uses VP9 in YouTube, Hangouts Meet and even Stadia. Google makes the decision of which codec to use based on the performance available in the machine in question, and at times, VP9 is preferred due to its better compression.

✅ Is hardware acceleration for video codecs important in WebRTC?

Yes and no.
WebRTC has succeeded to grow nicely with VP8 and virtually no mass hardware accelelration in devices. That said, vendors are preferring the use of hardware acceleration due to its advantages (battery life, less heating of the CPU, etc). This is why H.264 is the preferred codec by many vendors if/when they can use it. Remember tohugh that H.264 video codec use isn't a guarantee for availability of hardware acceleration.

✅ Is hardware acceleration for video codecs offer better performance than software video codec implementation?

Not necessarily. Hardware acceleration means certain features of the codec cannot be further optimized the same way that they can with software - features such as simulcast, machine learning based encoding or SVC are usually hard or impossible to achieve with hardware acceleration.
This makes the question of which codec implementation to use - software or hardware - a harder one to answer.

More about VP9 codec and WebRTC.

 Previous
Next

You may also like

Leave a Comment