OpenAI, LLMs, WebRTC, voice bots and Programmable Video
Learn about WebRTC LLM and its applications. Discover how this technology can improve real-time communication using conversational AI.
Read Moreadapter.js is the glue that sticks your code to the different browser implementations of WebRTC.
This article was co-written with Philipp Hancke. He has been the driving force behind adapter.js in the last two years, so it seemed like the best approach to have him contribute large portions of it. You can follow his writing here.
Need to know where WebRTC is available? Download this free WebRTC Device Cheat Sheet.
One of the visuals I created when I started out with WebRTC was this one:
It had several incarnations, and the main concept here is to show how WebRTC is different than traditional VoIP.
With traditional VoIP, you have multiple vendors implementing the specification, in hopes (as well as active interoperability checks) that the implementations will work in front of each other. If you knew one VoIP implementation, it said nothing about your ability to be able to yield another.
WebRTC was different. It brought to the table the concept of free, but also HTML5; and by that, I mean having a single API that every developer can use to add interactive voice and video to his application.
getUserMedia, PeerConnection and the data channel are all APIs specified in WebRTC. We’re now all speaking the same language when we’re implementing applications. And that, in turn, creates an ecosystem around it. One that was never there with such force with traditional VoIP.
Problem is, you can think of the WebRTC API as a suggestion only. That’s because today, version 1.0 of the specification isn’t yet a reality. We’ve got a candidate for it, but that says nothing about the implementations. Browser implementations of WebRTC are more like dialects of the same language. When you speak one, you understand another, but not fully. Not its nuances. And bad things can happen if two people with different dialects try to talk to each other without patience or understanding.
Which is probably where adapter.js comes into play.
Before we ask ourselves if adapter.js is needed today (it is), it would be worthwhile to understand how it came to be.
adapter.js has been around since the early days of WebRTC in late 2012 and early 2013. It was originally part of Google’s apprtc sample application. The original version can still be found in the Chrome tree. It was a very small project, less than 150 lines. The main job was to hide prefix differences like webkitRTCPeerConnection and mozRTCPeerConnection and to provide helper functions to attach a MediaStream to an HTML <audio> or <video> element.
During those wild west days of WebRTC, everyone wrote their own library to make WebRTC easier. This started to change in mid-2015 when Microsoft Edge came along. While Edge did not require prefixes for getUserMedia, attaching the MediaStream to a video element still worked in three different ways in as many implementations. This showed that there was a need to move to standardized behaviour. Also, as Microsoft’s Bernard Aboba pointed out, books were printed that showed the prefixed versions of the APIs -- which is the wrong thing to teach.
Preferring ORTC over the WebRTC 1.0 API, Microsoft was extremely happy to support the addition of a shim of the RTCPeerConnection API on top of ORTC. This enabled early interoperability and allowed ironing out some bugs before the first public ORTC-enabled Edge version.
A bit later, Promise support was added to adapter.js. Moving to Promises was one of the first big changes in the WebRTC specification and while Firefox has been adding them swiftly, Chrome was lagging behind. At that point, the “mission statement” for adapter changed. Instead of just trying to fill the gaps it became an enabler, allowing to write modern WebRTC Javascript. Mozilla’s Jan-Ivar Bruaroey recognized that and started contributing more elaborate pieces like a shim for the getUserMedia constraints.
When Safari started shipping WebRTC they contributed a shim for the “legacy” bits of the WebRTC API that they did not want to ship. This was an interesting attempt to get developers to write modern, promise-based WebRTC code. However, it does not seem to have worked out as sadly the release version shipped with the legacy API is enabled by default.
With growing complexity (currently over 2,200 lines of code) and being in the “hot path”, validation of changes to the adapter.s code itself became more of an issue. Initially powered by Selenium the scripts have been split up into unit tests and end-to-end tests that use standard tools like karma, mocha and chai to make assertions while running in a multitude of browsers on Travis-CI for every pull request and compare the results to previous runs. This shows the state of the art for testing WebRTC libraries and has been adopted by other projects as well.
During much of 2017, the main focus was on shimming the track-based API in Chrome. This is one of the bigger pieces of the move toward the WebRTC 1.0 API, described in this blog post by Mozilla and it was in adapter.js as well. The tests proved useful to ensure the consistency of the API which is particularly tricky since existing code might rely on certain interactions with the legacy API and that API (along with the interactions) is not specified. As is usual with large changes, there were a number of regressions -- however, it is much better to discover those regressions in a javascript library where the version can be pinned than to have Chrome ship them natively. Early in 2018, Chrome 64 will become stable and the native addTrack version will take over from the shimmed variant. Note: addTrack turned out not to be quite ready for production yet due to a bug related to getStats. The shim will continue to be preferred until Chrome M65 -- make sure your adapter version is updated after that change.
For a quick and dirty project you can simply include https://webrtc.github.io/adapter/adapter-latest.js in your code.
This will give you the latest published version. Note however that your application will automatically pull any changes so this is not recommended for larger applications.
The main source of adapter.js downloads is NPM. In most Javascript projects, you install webrtc-adapter as follows:
npm install webrtc-adapter
Note: Since adapter.js is manipulating the core WebRTC javascript APIs upgrading it is somewhat risky. Therefore it is recommended to keep the exact version specified in your package.json file and try it a lot when upgrading that version.
To use it, just require the module in one of your javascript files:
const adapter = require(‘webrtc-adapter’);
Since it is a polyfill, it transparently modifies the window object by default. The adapter object gives you information about the browser variant and version it detected in the browserDetails object:
console.log(adapter.browserDetails.browser);
console.log(adapter.browserDetails.version);
This is slightly different from a version detection library like platform as it treats Chromium-based browsers like Opera as Chrome -- since they run the same WebRTC engine that makes sense.
You can use the detected browser and version to add your own logic for working around bugs present in certain Chrome versions (e.g. the Chrome 61/Android video freeze or the Chrome 58 TURN/TCP issue).
To check WebRTC support you will need to check that RTCPeerConnection is defined:
!!window.RTCPeerConnection
and, if your use-case requires it, getUserMedia
!!(navigator.mediaDevices && navigator.mediaDevices.getUserMedia)
or the createDataChannel method of the RTCPeerConnection:
‘createDataChannel’ in RTCPeerConnection.prototype
After that you can simply write your WebRTC code as shown in the specification:
The official WebRTC samples are a great way to get started as they show a lot of use-cases and the maintainers ensure that they are semantically correct. Most of the shims are written in such a way that they will not become inactive when the native variant is available.
There are 4 forces at play with adapter.js:
Will a day come when we no longer need adapter.js?
Definitely.
But don’t wait up for it.
If the lifespan of jQuery is any indication (11 years and still going strong, with the last 4 of them with articles on why we don’t need jquery), we will be using adapter.js for many years to come.
Need to know where WebRTC is available? Download this free WebRTC Device Cheat Sheet.
Learn about WebRTC LLM and its applications. Discover how this technology can improve real-time communication using conversational AI.
Read MoreGet your copy of my ebook on the top 7 video quality metrics and KPIs in WebRTC (below).
Read More