OpenAI, LLMs, WebRTC, voice bots and Programmable Video
Learn about WebRTC LLM and its applications. Discover how this technology can improve real-time communication using conversational AI.
Read MoreWebRTC insights is turning out to be fun to create and super useful to our clients, looking to navigate the world of WebRTC.
Philipp Hancke and myself started this new thing called WebRTC Insights a year ago. We work well together, so we simply searched what we can do other than the WebRTC codelab, which was and still is a fun project.
WebRTC Insights is meant to help vendors sift through the technical (and non-technical) information that is out there and ever changing around WebRTC. Anything from bugs found, important changes in the WebRTC implementation to security issues raised and many other topics.
The idea? If you are a developer who uses WebRTC on a daily basis and relies on it, we can reduce the time you spend on finding what can bite you in the back when you weren’t looking. And we can definitely reduce the risk of that happening.
A year has gone by. The service evolved through this time, as we added more insights into it. Time to look at what we’ve done 😀
We started small. The first WebRTC Insights issue looked at 6 issues, 7 PSAs and 2 market insights. 4 pages in total. Now we’re at 15-20 issues on average (twice as much when a Safari release happened) and 10 pages (or more).
In numbers, over the year this turned out to be:
26 Insights issues, 331 issues & bugs, 120 PSAs, 17 security vulnerabilities, 74 market insights and 185 pages. Phew...
In the past decade we have had more than 13,000 issues filed against libwebrtc, Google’s implementation of WebRTC that we all use in Chrome (and all other browsers in one way or another), with close to 5,000 of them external bug reports. In addition to that close to 2,000 external chromium bugs related to WebRTC.
WebRTC is a complex piece of software and staying on top of it requires quite some effort. While the development activity on WebRTC is much lower these days (at a third of the peak change rate back in 2017) there is still a surprising amount of issues we have to look at.
WebRTC Insights started from conversations about WebRTC issues and the challenges they bring between us. We have long looked at and discussed bugs, but this happened over chat and we never wrote it up. Nowadays we write up a summary, our thoughts and the potential impact each bug has. Quite often we learn something from it.
In the process we actually created an annotated list of issues that we can then refer to when we encounter new issues. So when Tsahi complained about an increase in video jitter statistics recently, Philipp just pointed him to the issue where we discussed this topic (you see, Tsahi’s memory isn’t what it used to be).
“Public Service Announcements” or PSAs are a way for the WebRTC team (and Philipp) to communicate breaking changes in WebRTC. They range from changes to the C++ APIs to the plan-b deprecation and typically require action from developers using WebRTC in their applications.
We also list WebRTC-related Intent-to-ship from the Chromium process. This is a mandatory step in the process to launch WebRTC features that require Javascript API changes. In the last year we have mostly seen changes related to screen sharing which then turned into features of Google Meet - yet were available to other users of the platform as well.
Last but not least we do monitor the W3C working group and what happens there as it has a long term impact on where WebRTC is going.
WebRTC uses field trials in Chrome to roll out changes that have some technical risk. We identify them which gives us insights into what might be a possible root cause for issues that are hard to reproduce locally. The best example for this recently was this report by Facebook where an experimental change to reduce the noise during opus dtx caused a large AV desync issue. We had been tracking the experiment for a couple of weeks at that point.
We keep track of WebRTC related CVEs in Chrome (17 in the last twelve months), determine whether they only affect Chromium or when they affect native WebRTC and need to be cherry-picked into forks of the native library.
This part is the bird’s eye view that we offer. The rest of the insights are the low level details developers need. Here, we look at the bigger picture of what WebRTC is and the market forces around it.
We bump into tweets, posts, LinkedIn messages and other articles out there - and when we feel they are relevant and important to your work, we mention them. And explain where we see this trend headed and what you should be aware of.
The market insights are designed and handpicked for the clients we serve in WebRTC Insights.
Over time, we’ve evolved the service.
Security and Chrome trials were added later on. We are now experimenting ourselves with short video explainers of each libwebrtc release (=once a month) and its implications to developers. We got some great feedback on it, so we’re likely to keep it as part of our format.
There are now also 3 different plans to the WebRTC Insights:
Want to join us for the ride this coming year?
👉 To learn more, check us out at WebRTC Insights
👉 You can leave us a message there to get a sample copy of one of our latest insights issue
Learn about WebRTC LLM and its applications. Discover how this technology can improve real-time communication using conversational AI.
Read MoreGet your copy of my ebook on the top 7 video quality metrics and KPIs in WebRTC (below).
Read More