Kranky Geek WebRTC event summary 2022

Kranky Geek 2022 follows our tradition of great curated content on WebRTC that is both timely and timeless. Here’s what we had this year.

Kranky Geek is the main event focusing on WebRTC. I’ve been doing it with Chris Koehncke and Chad Hart for many years now, with the help and assistance of Google along with various sponsors each time.

Like many, we’ve switched to an all virtual event since the pandemic started, and decided at least for this year to continue in the same format. This turned out well, since I had to go on a business trip to Ireland at the date of the event, and virtual meant I was still able to both host and speak at the event.

Kranky Geek is quite a grueling experience for the hosts. We curate the sessions, at times approaching those we want to speak, at other times telling the speakers what topics we think will fit best. We go over the draft slide decks and comment on them. Doing dry runs on the week of the event with all speakers to make sure the session is top notch.

You won’t find much commercial content in a Kranky Geek event. What you will find is lots of best practices and suggestions based on the experience and the path taken by our great speakers.

To this year’s summary, Philipp Hancke did the commentary about the sessions themselves. If you are a WebRTC Insights subscriber, and would like to discuss the content and how it fits in your company, feel free to reach out to me to schedule a meeting.

If you are looking for the whole playlist, you can find it here. The videos have been embedded below to make it easier for you to watch.

Roundtable: The state of Open Source in WebRTC

https://www.youtube.com/watch?v=f4nfoXhfTv0&list=PL4_h-ulX5eNf-ZShwxvzd6jTyFagwjh0C&index=2&ab_channel=KrankyGeek

• Jitsi, Janus, mediasoup and Pion
• Watch if you are using any of these projects

AI in Google Meet / Dan Gunnarsson, Google

Background blurring and light adjustment using MediaPipe.

https://www.youtube.com/watch?v=k92zfj4wUVc&list=PL4_h-ulX5eNf-ZShwxvzd6jTyFagwjh0C&index=3&ab_channel=KrankyGeek

• ML powered background blur and light adjustment using MediaPipe
• Performance in the browser is a challenge, as is model size
• A lot of data and practical examples. Not an introduction to MediaPipe though
• Watch after you built a background blur pipeline yourself and want to learn how to improve it

Performant Real Time Audio ML in the Browser / Arman Jivanyan, Krisp

Krisp SDK on the Web: noise suppression.

https://www.youtube.com/watch?v=nwONF2lvovw&list=PL4_h-ulX5eNf-ZShwxvzd6jTyFagwjh0C&index=4&ab_channel=KrankyGeek

• ML powered audio improvements on the Web
• 128 samples per frame suggest using WebAudio / Audio Worklets. Note that there’s a Chrome “L16 hack” which can deliver 10ms frames of “raw” audio
• Watch when you consider building an audio processing pipeline yourself as well as after your first attempt

Making sense of WebRTC statistics / Tsahi Levent-Levi, Spearline/testRTC

Where I speak and Philipp comments (I am kinda subjective on this session).

https://www.youtube.com/watch?v=kpd9kApUYdM&list=PL4_h-ulX5eNf-ZShwxvzd6jTyFagwjh0C&index=5&ab_channel=KrankyGeek

• WebRTC’s getStats API is tremendously powerful but making sense of the numbers is quite a challenge. Turning those numbers into something actionable is quite tricky
• There are a lot of things that can go wrong and are actionable. The tooling testRTC has is quite valuable answering the common support questions. Troubleshooting is important but may depend on how much time you can spend on debugging a customer's environment
• Watch when you are stuck trying to troubleshoot a problem with just WebRTC statistics and need to take a look beyond them

WebRTC annual update 2022 / Google

https://www.youtube.com/watch?v=WEUb-G4oSAg&list=PL4_h-ulX5eNf-ZShwxvzd6jTyFagwjh0C&index=6&ab_channel=KrankyGeek

• A billion minutes every day. That sounds impressive. There is a but here however. In 2018 the number Google told was 2.5 billion per week or around 400 million per day (on weekdays and half on weekends?). That means a 2.5x growth in four years. With a pandemic in between. Meta said Whatsapp is doing 15 billion minutes per day… WebRTC in the browser remains small. One wonders what happened to their 100x usage (which was received minutes). We’re past peak usage
• Some Insights into their roadmap. We have been tracking most of this in WebRTC Insights over the past year so no big surprises if you are a subscriber
• Elad Alon provides a good overview of the improvements he did to screen sharing such as preferring tab sharing in the getDisplayMedia picker
• Markus Handell talks about a lot of things like Metronome which, as a reader of WebRTC Insights may mean something to you. Great slide that shows the pipeline and where the improvements to individual components affect it
• Harald Alvestrand gives a great summary of what new APIs are coming to WebRTC in general. Control about ICE candidates seems super interesting but we have not yet figured out what the field trials is actually enabling

Compositing in the cloud with native pipelines / Pauli Ojala, Daily

Recording and compositing video sessions.

https://www.youtube.com/watch?v=YZuB4Fo7r4k&list=PL4_h-ulX5eNf-ZShwxvzd6jTyFagwjh0C&index=7&ab_channel=KrankyGeek

• Great overview of the considerations you need to make when doing WebRTC recording
• The goal is to get a view like in the browser but you cannot record in the browser, not even on the server
• Watch if you are looking to build a recording feature and are interested in the requirements

WHIP and WHEP: Standardized Live Streaming with WebRTC / Sergio Garcia Murillo, Dolby

https://www.youtube.com/watch?v=rIQVVJOjR0U&list=PL4_h-ulX5eNf-ZShwxvzd6jTyFagwjh0C&index=8&ab_channel=KrankyGeek

• A great introduction to why streaming wants to have “standardized signaling”
• WebRTC was built with JavaScript and the flexibility in mind but native applications need a bit more standardization. And let us not talk about hardware encoders
• In many ways, WebRTC breaks the model of vendor’s ecosystem where media servers and device manufacturers of the past had to interoperate by having a single vendor take care of it all. For live streaming and broadcasting, this interoperability model is still very important
• Watch if you are interested in streaming use-cases

Using Video Forward Error Correction to improve game streaming quality / Harsh Maniar, NVIDIA

FlexFEC and video.

https://www.youtube.com/watch?v=igm7QkqxHqk&list=PL4_h-ulX5eNf-ZShwxvzd6jTyFagwjh0C&index=9&ab_channel=KrankyGeek

• NVIDIA uses video forward error correction for GeforceNOW, their game streaming service. The requirements for such a service are quite different from WebRTCs “talking heads”
• Video forward error correction is a surprisingly obscure topic in WebRTC
• Watch if you are interested in learning how FEC works and how to measure the improvements

Advances in audio codecs / Philipp Hancke

https://www.youtube.com/watch?v=l94DITPcmn0&list=PL4_h-ulX5eNf-ZShwxvzd6jTyFagwjh0C&index=10&ab_channel=KrankyGeek

• Audio remains the most important thing in conferencing - you need to understand what the other side is saying. This talk explains the history of Opus and how Lyra and Satin fit into the picture as well as how forward error correction and redundancy work
• The famous “Opus comparison” picture is quite problematic when looked at in detail
• Watch if you are interested in going beyond the usual Opus forward error correction mechanism available in WebRTC by default

Our Kranky Geek sponsors

It should be noted that without our sponsors, doing the Kranky Geek event would be impossible. When we set out to run these events, we had this in mind:

• Content should be free for all
• Participating in the event should be free or have a token cost associated with it
• Content must be top notch. Timeless. With little to no sales pitches

This requires sponsors to help with funding it. Each year we search for sponsors and end up with a few that are willing and happy to participate in this project of ours.

This year?

• Google, and especially the team working tirelessly on WebRTC
• Daily, who operates a video API (CPaaS) platform with a strong lowcode/nocode focus
• Krisp, with their AI solution to handle background voices, noises and echo
• Spearline, and their testRTC products for WebRTC testing and monitoring

Check them out 😉

A Kranky Geek 2023?

When we’re planning and preparing for the event, it feels like this is going to be our last event. It isn’t easy, and none of us in the Kranky Geek team are event planners by profession. The question arises after each such event - will we be doing another one?

Once this event was over, we started working on wrapping the event. Part of it was editing the content and uploading it to YouTube (which takes time).

Will there be another Kranky Geek event next year? Maybe

Will it be in person or virtual? Maybe 😆

Until then, go check out our growing library of great WebRTC content: https://www.youtube.com/krankygeek