WebRTC and Server GPUs? A whitepaper

GPUs is most probably where we're headed. A couple of months ago, I was approached by SURF. An Israeli vendor specializing in server media processing. As many of its peers, SURF has been migrating from hardware based DSP systems to software systems in their architecture. As they've entered the WebRTC space, they wanted to have a whitepaper on the topic, and I accepted the challenge. The end result? WebRTC Server Side Media Processing: Simplified

Two things that I wanted to share here:

#1 - WebRTC Server Side Media Processing is real

What made writing this whitepaper so interesting for me was the fact that there really is a transition happening - not to using WebRTC - that already happened as far as I can tell. It is something different. A transition from simple WebRTC services that require a bit of signaling to services that process the media in the backend. This processing can be anything from recording to gatewaying, streaming, interoperating or modifying media in transit. And it seems like many commercial use cases that start simple end up requiring some kind of server side media processing. In the span of the last two months, I've seen quite a few services that ended up building some WebRTC server side media processing for their use case. Maybe it is just related to the research I did around this are for the whitepaper, but I think it is more than that.

#2 - The Future Lies in GPUs

As I was working on the whitepaper, this one got published by Jeff Atwood - it is about AI winning a game of Go. Or more accurately, how GPUs are a part of it:

GPUs are still doubling in performance every few years

The whole piece is really interesting and a great read. It also fits well with my own understanding and knowledge of video compression (=not that much). Two decades ago, video compression was a game of ASIC - the ugliest piece of technology possible. Hard to design and develop. You wanted to implement a new video codec? Great. Carve a few years for the task and a couple of millions to get there. They are hard to design and hard to program for. Later it was all DSP. Still hard and ugly, but somewhat cheaper and with some flexibility as to what can get done with them. DSPs is what powers most of our phones when it comes to recording and playing back videos. It works pretty well and made it seem as if the device in our pocket is really powerful - until you try using its CPU like a real PC. GPUs were always there, but mostly for gaming. They do well with polygons and things related to 2D and 3G graphics, but were never really utilized for video compression. Or at least that's what I thought. I heard of CUDA in passing. Heard it was hard to program for. That was something like 5 years ago I believe. Then I read about GPUs being used to break hashes, which was an indication of their use elsewhere. The Jeff Atwood piece indicated that there are other workloads that can benefit from GPUs. Especially ones that can be parallelized, and to some extent, video compression is such a task. It is also where SURF is focusing with its own server media processing, which places them in the future of that field. GPUs are no longer used only for gaming or in our PCs and laptops - they are also being deployed in the cloud. They assist companies running Audocad in the cloud (heard such a story in the recent WebRTC Global Summit event), so why not use them for video compression when possible? - If you are interested in WebRTC and how media processing is finding its way to the server, and how that fits in with words like cloud and GPU, then take a look at this new whitepaper. I hope you'll enjoy reading it as much as I've enjoyed writing it. Download and read this new WebRTC whitepaper.