Can I trust WebRTC getStats accuracy?

Yes and no. WebRTC getStats is what we have to work with, so we have to make do with it. That said, your real problems may lie elsewhere altogether.

👉 Philipp Hancke assisted in writing this article and Midjourney helped with most of the visuals

This is the question I was posed in a meeting last week:

Can I trust WebRTC getStats?

As the Jewish person that I am, I immediately answered with a question of my own:

Assume the answer is “No". What are you going to do now?

I thought the conversation merits a bit more discussion and some public sharing, which led to this article being written.

TL;DR

Yes. You can and should trust the accuracy of WebRTC getStats, but like with everything else, you should also keep a dose of happy suspicion around you.

Like any piece of software, libwebrtc and its getStats implementation by extension, has bugs. These bugs get fixed over time. The priority given to fixing them relates mostly to how much Google’s own services suffer from and a seemingly arbitrary prioritization for the rest of the issues.

See below to learn more on why we have a problem and what you can do about it.

A short history of WebRTC getStats

WebRTC was announced somewhere in 2011 and the initial public code in Chrome was released in 2012. The protocol itself was stabilized and officially published by the W3C in January 2021. Just… 10 years later.

In between these 10 years a lot of discussions took place and the actual API surface of the WebRTC standard specification was modified to fit the feedback provided and to encompass additional use cases and requirements.

We’ve had these discussions taking place in parallel to WebRTC being implemented in web browsers and shipped out so developers can make use of them. Years before WebRTC was officially “standardized” we had hundreds if not thousands of applications in production using WebRTC, oftentimes with paying customers.

At some point, the getStats implementation in the standard specification diverged from that implemented by Google in Chrome, ending with two main alternatives:

1. Spec-compliant getStats - the new API that adheres to the standard specification. Given that this specification is authored by Googlers it is not surprising that it ended up being a description of what Chrome implemented, whether it made much sense or not. This was added in Chrome 58 back in January 2017
2. Legacy getStats - the original implementation in Chrome

. This made switching from one to the other a challenge:

• Google could just implement the new stats, but that would break applications that used legacy getStats implementation
• Developers wanted to use the spec compliant stats, but needed a browser that supports them

The decision was made that the distinction between the two would be how getStats() is called. Callback-based invocation returned the legacy stats while using a promise returned spec-compliant getStats. The logic behind this was that promises was a new construct introduced to Javascript at the time, so developers who used the legacy getStats didn’t use promises (yet).

This approach worked rather well for the last 6 years, with many (most?) applications adopting the use of the spec-compliant getStats:

We observed a step drop in usage when Google Meet stopped using the legacy API (that’s the blue line going down). That said, a few outliers still remain who use the old getStats. They will not be able to do so in 2024.

Google WebRTC housecleaning project

Fast forward to today (or last year).

WebRTC is a solid standard and implementation used by many. It got us through the pandemic in many ways and aspects.

All the bigger requirements from WebRTC are behind us. There aren’t that many innovations or new features that get introduced to it.

Which is leading Google in recent months to house cleaning tasks:

• Figuring out where they can squeeze the lemon a bit more for performance reasons
• Where they can get rid of deadweight by deprecating and killing unnecessary code
• Following the WebRTC specification even more closely
• Beefing up best practices in security even more

This house cleaning work has reached getStats, and with it, 4 main areas:

1. Deprecating and later killing legacy getStats (after waiting for Google Meet to stop using it and migrate to the spec compliant variation)
2. Trimming down the results object for performance reasons
3. “Randomizing” the object identifiers in the returned getStats structure for both performance and best practices reasons. This is still planned so it is best to prepare for it and not to interpret the “id” attribute in any way
4. Making sure all stats in the specifications are reflected in the getStats implementation itself

Such changes are great when viewed in the long term. But in the short term they are a huge headache.

Firefox & Safari

Since Safari uses libwebrtc, it will get most statistics out of the box. However, the binding at the WebKit layer needs some code to be written which creates some difference with libWebRTC changes that Safari does not notice. We observed this with the “trackIdentifier” property recently but there may be others. Apple seems rather reactive here.

Firefox used to spearhead the “spec” getStats implementation but has fallen behind and lacks several stats types (such as candidate-pair stats). This means workarounds like shown by this WebRTC sample are still required for very basic functionality. Statistics related to media quality are lacking even more.

Keeping up the pace with WebRTC getStats changes

At testRTC, we’re offering tools for the full lifecycle of WebRTC applications. These include testing and monitoring services. As such, we rely heavily on getStats.

Years ago, we had to implement the migration from legacy stats to spec compliant stats.

Then came 2022 and with it the housekeeping changes by Google to the statistics found in getStats. It started with Chrome 107 and continues even today. With each such release, we need to get an experienced WebRTC developer to check, test and fix our code to make sure our services collect the statistics properly. All that is on top of the need to support more metrics that Google adds to Chrome in WebRTC getStats from time to time.

Our job is harder than most in this simply because we need to collect and support all the stats - the customer base we have is varied and we never really know which metrics they’d be interested in.

This task of keeping up with getStats has been a bit of a challenge in the last few months. That’s because in each release something else changes. Each step is reasonable. Needed. Minor. But it brings with it changes we need to do in our own planning and roadmap.

To others, such changes have brought with them breakages as well. At times the need to update and upgrade open source components or to fix their own code.

This is a good thing

It is important to state - the changes and work conducted here by Google is for the better.

Going for a spec compliant WebRTC getStats implementation means we have actual documentation that we expect to work. It also means interoperability with other browsers and components (assuming they strive to spec compliance as well).

Improvements in performance and polishing out best practices means better performance and code for WebRTC applications in general.

Removing deadweight and deprecated/unused statistics and similar components means smaller codebase with less edge cases and “things” to test.

This is what we want our WebRTC implementation to be and look like.

The fact that we need to undergo this ordeal is the price we need to pay for it. It would have been a wee bit nicer if Google would lay their plans of such changes well in advance (not through sporadic PSAs but rather as a kind of a public roadmap). This will enable better planning for those running such applications. But it is what it is. And frankly - we get what we pay for (=free).

Chrome’s WebRTC getStats implementation might not be the reason for bad metric values

Then there are bugs. Metrics you obtain for getStats that don’t seem to reflect reality.

There are usually 3 reasons for that to happen:

1. Chrome. There’s a Chrome bug that leads to bad metrics results via getStats. As I stated earlier, these get fixed based on the priority and backlog of Google when it comes to their libwebrtc library
2. You. The value is correct. You just don’t understand what it means or how it gets calculated. Since there’s little in the way of documenting each and every metric in getStats, this is quite common
3. The other side. When your browser interacts with a non-browser device, a native mobile application or a media server, it gets a lot of the data used to report specific metrics via WebRTC getStats from RTCP reports that are calculated, generated and sent by the other device. That side may also have bugs in it (highly likely and even more)

A few things to remember here:

👉 WebRTC is used by MANY inside browsers. Think billion(s) of people

👉 It is adopted by thousands of applications developing directly and indirectly on top of it

👉 Using statistics is standard practice to optimizing for media quality and most of the large WebRTC applications rely on it heavily already

❓ Why should your application and use case be any different in trusting WebRTC getStats?

What can you do about WebRTC getStats changes?

Nothing.

That said, I do have a few suggestions for you:

1. Understand and assume that things will change, bugs will be found (and fixed), and that for the most part, getStats is a really powerful and useful tool
2. Test your application (and its stats) against the latest browser builds. This should include the upcoming beta and even the nightly builds if you’re up for it
3. Make sure your media servers and other components are up to date. Especially in the RTCP reports they spew. When in doubt, question their behavior before libwebrtc (remember that they also need to run after Google’s implementation of WebRTC in Chrome)
4. Subscribe and follow the WebRTC Insights. That’s where we flag such upcoming issues, among other things we cater for 😉