OpenAI, LLMs, WebRTC, voice bots and Programmable Video
Learn about WebRTC LLM and its applications. Discover how this technology can improve real-time communication using conversational AI.
Read MoreIt is time to stop for a second and review what we’ve accomplished here with our WebRTC Insights in the past two years.
There are a few pet projects that I am doing with partners, and one of the prime partners in crime for me is Philipp Hancke. We’ve launched our successful WebRTC codelab and are now in the process of finalizing our second course together - Low-level WebRTC protocols.
Two years ago, we decided to start a service - WebRTC Insights - where we send out an email every two weeks about everything and anything that WebRTC developers need to be aware of. This includes bug reports, upcoming features, Chrome experiments, security issues and market trends.
All of this with the intent of empowering you and letting you focus on what is really important - your application. We take care of giving you the information you need quicker and in a form that is already processed.
Now, two years in, it is safe to say that this is a VERY useful tool for our subscribers.
“WebRTC insights might be the most important email you read every fortnight as a RTC / video engineer. It’s hard to keep tabs on what Google et al are doing with WebRTC while working on your product and the WebRTC Insights provides very specific and actionable items that help tremendously. We have been ahead countless times because of it. If you are serious about WebRTC you should definitely subscribe 100% worth it.”
— Saúl Ibarra Corretgé, Principal Software Engineer @ 8x8 (Jitsi)
Keeping track of all the changes in WebRTC is a pretty daunting task. Tsahi started WebRTC Weekly almost nine years ago and it has been the source of high-level information ever since. Philipp has closely worked with WebRTC at a more technical level for a decade too. We both had our routines for keeping notes and transforming them into something informative for our audience but joining forces (which we never expected after having strong arguments about whether XMPP was a great signaling protocol in the early days!) has yielded a surprising amount of synergy effects.
We start doing Insights with a template. Whenever we find something that we think is interesting we add a link and maybe a very brief comment to that template . Usually we chat about those too (as we have done for…. almost a decade now). Then we move on because both of us have day jobs that keep us busy.
Every two weeks we spend a couple of hours turning the “brain dump” into something that our audience understands. Philipp focuses on the technical bits while Tsahi focuses on the market. Then we review each other's section, improve and exchange thoughts.
We did this before Insights already but putting a structure and a biweekly cadence to it has “professionalized” it. While it remains a side project for us, we now have the process in place.
We’re not new to this, as this is our second year, we might as well also compare the numbers today with those we’ve had on year one of WebRTC Insights:
26 Insights issued this year with 447 issues & bugs, 151 PSAs, 11 security vulnerabilities, 146 market insights all totalling 239 pages. We’ve grown on all metrics besides security vulnerabilities.
WebRTC is still ever changing, but at least there are less security threats in it 😉
Activity on libWebRTC has cooled down a bit in the last two years when it comes to the number of commits and people working on it:
After more than a decade that is a sign of maturity, the easy changes have already been done and all that is left is optimizations. The numbers we see for Insights roughly correlate with the amount of energy Google puts into the project. We are just glad we did not start it during the “hot phase” of 2016-2019.
Let’s dive into the categories, along with a few new initiatives we’ve taken this year as part of our WebRTC Insights service.
Among the really useful feedback we have received was the suggestion to add a “component” or area the issue is in. This is useful for larger teams where one person may be digesting the biweekly email and route this to a subteam with a particular focus such as audio, video or networking.
The other improvement is a visual hint whether a particular item is a bug, a regression, a feature or just something that is generally good to know:
In addition to that we classify it as “read, plan or act”. Of course we hope our subscribers read all the issues but some are more important than others.
Public service announcements or PSA are the main method Google’s WebRTC team uses to announce important changes on the discuss-webrtc mailing list. We track them and give some context why they are important or whether they are safe to ignore (which can happen for API changes where a PSA may be required by the release process.
We also look at important W3C changes in this section as well as other content that is too technical for the “market watch” section.
Chrome’s field trials for WebRTC are a good indicator of what large changes are rolling out which either carry some risk of subtle breaks or need A/B experimentation. Sometimes, those trials may explain behavior that only reproduces on some machines but not on others. We track the information from the chrome://version page over time which gives us a pretty good picture on what is going on:
In this example we saw the AV1 decoder switch from libaom to libdav1d over the course of several weeks.
This year we continued keeping track of WebRTC related CVEs in Chrome (totaling 11 new ones in the past year). For each one, we determine whether they only affect Chromium or when they affect native WebRTC and need to be cherry-picked to your own fork of libwebrtc when you use it that way.
To make it easier to track, we now keep a separate Security Tracker file that gets updated with new issues as they are found. This makes it easier to glance at all the security issues we’ve collected.
On top of that, when there’s a popular open source component that has its own security issues published, we tend to also indicate these, though not add them to the Security Tracker, so they aren’t even counted in our statistics.
Information overload. That’s what all of us face these days with so much material that is out there on the Internet. On our end, we read a lot and try to make sense of it.
Part of that is taking what feels relevant to WebRTC and sharing it with our WebRTC Insights subscribers. It includes the reference to the article, along with our thoughts about it.
For product managers, this is their bread and butter in gleaning the bits and pieces of information they need to make educated decisions about roadmap and priorities.
For developers, this brings a bit more context than they are used for in their daily work - and is often outside of their immediate work and expertise.
Our purpose? Enrich your world about WebRTC and express some of the power plays and the shifts in the market that are taking place. So you know them well ahead of them happening in force.
We really enjoyed Meta’s RTC@scale event. In terms of quality and technical depth it set a bar for the upcoming KrankyGeek event which had been the gold standard so far.
However, the technical depth of the event was too intense for it to be digested in real-time. This meant Philipp sat down on a rainy Saturday and started rewatching the videos while keeping notes. And ended up watching each session multiple times since there were so many great points that needed or even demanded a bit more explanation. This turned into a nine page summary of the event, annotated with the timestamps in the video.
We decided to make this summary public because, while we thought it provided a ton of valuable lessons to our subscribers. Meta made the content freely available and so should we. And hey, we keep referencing this every other week.
This may have been a one-off but we still genuinely enjoyed it so might repeat the exercise… on a rainy saturday!
We started playing around with video release notes at the end of our first year, and quickly made it a part of the WebRTC Insights service.
Whenever Google publishes a release notes for WebRTC, we publish our own video with a quick analysis of the release notes (and the release itself) for our Insights clients.
We go over the release answering 4 main questions:
Our intent here, as with anything else, is to reduce the amount of work our clients have to do figuring out WebRTC details.
We are also making these release notes videos publicly available, 3-4 versions back, so you can derive value from them. You can find them on YouTube:
https://www.youtube.com/watch?v=DQt_OQT4ZAo&list=PL7fuFATIj-PUtMVTQKpW_odTCO0_CPfXV
Be sure to subscribe to receive them once they get published freely to everyone.
We are now headed into our third year of WebRTC Insights.
Our number of subscribers is growing. If you’ve got to this point, then the only question to ask is why aren’t you already subscribed to the WebTRC Insights if WebRTC interests you so much?
You can read more about the available plans for WebRTC Insights and if you have any questions - just contact Tsahi.
👉 Oh - and you shouldn’t take only our word for how great WebRTC Insights - just see what our readers have to say about it:
"For any Service Provider or Apps who heavily relies on WebRTC, the WebRTC Insights offers great value. [...] What I like most about the Insights is its bi-weekly cadence, which fits the rapid Chrome/WebRTC release cycle, and most of the mentions are actionable for us. With the recent Safari audio breakage, the Insights highlighted the problem timely and saved us a lot of troubleshooting effort."
— Jim Fan, Engineering Director @ Dolby Laboratories
“As a service company specialized in WebRTC I think WebRTC Insights is really useful. It keeps us up to date about what is coming next, giving good ideas for projects and research. Also, receiving periodic insights is always a good excuse to stop what I am doing and find some time to go over the latest WebRTC updates in more detail. It is much easier to do when you get all summarized in a single document than on your own just googling and going through an overwhelming list of webrtc news, updates and bugs.”
— Alberto Gonzalez Trastoy, CTO @ WebRTC.ventures
👉 Here’s the summary of the first year of Insights if you’re interested
Learn about WebRTC LLM and its applications. Discover how this technology can improve real-time communication using conversational AI.
Read MoreGet your copy of my ebook on the top 7 video quality metrics and KPIs in WebRTC (below).
Read More