WebRTC iOS 11 Support. Are We There Yet?

Last week WWDC was a happy occasion for those who deal with WebRTC. For the first time, we got the official word - and code - of WebRTC on Apple devices - WebRTC iOS and WebRTC Safari support is finally here.

I spent the time since then talking to a couple of developers and managers who either tinkered with Safari 11 already or have to make plans in their products for this development.

I came out a bit undecided about this whole thing. Here are where things stand.

Apple’s Coverage

The WebKit site has its own announcement - as close as we’ll ever get to an official announcement from Apple with any detail it seems.

What I find interesting with this one (go read it - I’ll wait here):

• Google isn’t mentioned (god forbid). But their “LibWebRTC open source framework” is. Laughed my pants off of this one. The lengths companies would go to not mention one another is ridiculous
• WebRTC in WebKit isn’t mentioned. Two years ago I opined it wouldn’t move the needle. And it seems like it didn’t - or at least not enough for Apple to mention it in this WebKit announcement
• TokBox and BlueJeans are mentioned as early beta. TokBox I understand. Can’t miss one of the dominant players in this space. But BlueJeans? That was a surprise to me

Previous Coverage

First things first, here are some posts that were published already about Apple’s release of WebRTC Safari (in alphabetical order):

• Peer5 - great for the industry. And how it makes sense when Flash is dying and an alternative low latency solution is needed in the browser
• Quobis - checked their Sippo status. Interesting to see what works and doesn’t “out of the box”
• The New Dial Tone - hold on with the party. Amir Zmora details what’s included in Apple’s WebRTC as we know it now
• TokBox - excited about WebRTC in iOS 11 and Safari. Mostly due to the fact that this solves spontaneous use cases
• WebRTC.is - WebRTC ships in MacOS and iOS 11. Eric Lagerway wakes up after a year of non-activity on his blog
• WebRTC.ventures - WebRTC support in Safari 11. Arin Sime weighs in on what to expect in the short-mid term (=nothing yet), but sees a bright future

I’ve ignored a few generic posts that just indicated WebRTC is out there.

Most relevant Twitter thread on this?

https://twitter.com/saghul/status/872066176876195845

Who’s Missing?

Well… the general media.

I haven’t really seen the keynote. I find the Apple keynotes irrelevant to my needs, so prefer not to waste my time on them. My understanding is that WebRTC took place there as a word in a slide. And that’s HUGE! Especially considering the fact that none of the general technology media outlets cared to mention it in any way.

Not even the general developers media.

What was mentioned instead were things like the iOS file manager. That seems a lot more transformative.

This isn’t a rant about general media and how they miss WebRTC. It is a rant about how WebRTC as a technology has not caught the attention at all, outside a little circle of people.

Is it because communications is no longer interesting?

At a large developers event here in Israel on that same week, there were 6 different tracks: Architecture, Full stack, Philosophy, Big Data, IOT, Security. No comms.

Not sure what the answer…

On one hand, WebRTC is transformative and important. On the other hand, it is mostly ignored.

WebRTC Safari Support

So what did get included in the WebRTC Safari implementation by Apple?

• The basics. GetUserMedia and PeerConnection
• One-on-one voice and video calls seem to work well across browsers AND devices (including things like Safari to Edge)
• Opus is there for audio
• H.264 only for video. There is no VP8 or VP9. More on that later
• The code supports Plan B, if you are into multiparty media routing
• Data Channel is supported, but too buggy to be used apparently
• No screen sharing. Yet
• This is both for the desktop and mobile:
  • WebRTC Safari support is there for the macOS in the new version of safari
  • WebRTC iOS support is there for iOS 11 - via the Safari web browser, and maybe(?) inside WebViews

This is mostly expected. See below why.

WebKit, Blink and WebRTC

A long time ago, in a galaxy far far away. There was a browser rendering engine called WebKit. Everyone used WebKit. Especially Google and Apple. Google used WebKit for Chrome. And Apple used WebKit for Safari.

And then one day, in the middle of the development of WebRTC, Google decided enough is enough. They forked WebKit, renamed it to Blink, removed all excess code baggage and never looked back.

Apple never did care about WebRTC. At least not enough to make this thing happen until last week. I hope this is a move forward and a change of pace in Apple’s adoption of WebRTC.

Here’s what I think Apple did:

• Seems like they just took the Google code for WebRTC and hammered at it until it fit nicely back into WebKit (ignoring WebRTC in WebKit in the process)
• How did they modify it? Remove VP8. Add H.264 by hooking it up with the hardware codec in iOS and on the Mac
• And did the rest of the porting work - connecting devices, etc.
• Plan B is there because. Well. Google uses Plan B at the moment. And it just stands to reason that the code Apple had was Plan B

WebRTC Safari and Interoperability

When it comes to WebRTC, the question is one of browser interoperability. There aren’t many browser vendors (I am counting four major ones).

The basics seem to work fine. If you run a simple peer-to-peer call between any of the 4 browsers, you’ll get a call going. Voice and video. The lowest common denominator for that seems to be Opus+H.264 due to Safari. Otherwise, Opus+VP8 would also be a possibility.

The challenge starts when what you’re trying to do is multiparty. While H.264 is supported by all browsers, the ability to use simulcast with H.264 isn’t. At the moment, Chrome doesn’t support simulcast with H.264. The current market perception is that multiparty video without simulcast is meh.

Doing group calling?

• Go for audio only
• Force everyone to use H.264 if you need Safari (either as a general rule or when the specific session has someone using Safari) - and understand that simulcast won’t be available to you

Now it is going to become a matter of who blinks first: Google by adding H.264 simulcasting to Chrome; or Apple by adding VP8 to Safari.

The Next Video Codec War

Which leads us to the next frontier in the video codec wars.

If you came in late to the game, then know that we had over 3 years of continuous bickering regarding the mandatory video codec in WebRTC. Here’s the last I wrote about the codec wars when the Alliance of Open Media formed some two years ago.

At the time, both VP8 and H.264 were defined as mandatory video codecs in WebRTC. The trajectory was also quite obvious:

After  H.264 and VP8, there was going to be a shift towards VP9 and then a leap towards AV1 - the new video codec currently being defined by the Alliance of Open Media.

Who isn’t in the alliance? Apple.

And it seems that Apple decided to bank on HEVC (H.265) - the successor of H.264. This is true for both iOS and macOS. This is done through hardware acceleration for both the encoder and the decoder, with the purpose of reducing storage requirements for video files and reducing bandwidth requirements for streaming.

But it goes to show where Apple will be going with WebRTC next. They will be adding HEVC support to it, ignoring VP9 altogether.

There’s a hefty cost in taking that route:

• H.264 is simple yet expensive - when you use it, you need to pay up royalties to a single patent pool - MPEG-LA
• HEVC is complex AND expensive - when you use it, you need to pay up royalties for MULTIPLE patent pools - MPEG-LA, HEVC Advance, Velos Media. Wondering which one you’ll need to pay for and how much? Me too

Which is why I think Apple is taking this route in the first place.

Apple has its own patents in HEVC, and is part of the MPEG-LA patent pool.

And it knows royalties is going to be complex and expensive. Which makes this a barrier for other vendors. Especially those who aren’t as vertically integrated - who needs to pay royalties here? The chipset vendor? The OS vendor? The handset manufacturer?

By embedding HEVC in iOS 11 and macOS High Sierra, Apple is doing what it does best - differentiates itself from anyone else based on quality:

• It has hardware acceleration for HEVC. Both encoding and decoding
• It starts using it today on its devices, and “magically” media quality improves and/or storage/network requirements decrease

Google, and Android by extension, won’t be adding HEVC. Google has taken the VP9 route. But in VP9 most solutions are software based - especially for the encoder. Which means that using VP9 eats up CPU. Results look just as good as HEVC, but the cost is higher on CPU and memory needs. Which means an “inferior” solution.

Which is exactly what Apple wants and needs. It just doesn’t care if interoperability with other devices requires lowering quality as the perception of who's at fault will fall flat on Android and Google and not on Apple.

https://www.youtube.com/watch?v=8IKxOIbRVxs

So…

Don’t expect to see VP9 or AV1 anytime soon in Apple’s roadmap. Not for WebRTC and not for anything else.

Dan Rayburn covers the streaming side (non-WebRTC) of this HEVC decision quite nicely on StreamingMedia.

Oh, and while at it, Jeremy Noring wrote a well thought rant about this lack of support for VP8 and VP9. His suggestion? Go vote for bug #173141 on WebKit. It probably won’t help, but it will make you feel a bit better :-)

The only way I see this being resolved? If Google retracts their support for H.264 and just blatantly removes it from Chrome until Apple adds VP8 to Safari. I’ll be happy to see this happening.

FaceTime , Multiparty and WebRTC

Apple has FaceTime.

And FaceTime probably doesn’t use WebRTC. I am not sure if it ever will.

When Google came out with WebRTC, they had the Hangouts (now Meet) team about a year behind in its adoption of WebRTC as their underlying technology - but the intent and later execution was there.

When Microsoft came out with WebRTC, Skype didn’t support WebRTC. But they did launch Skype for Linux which is built somewhat on top of WebRTC, and Skype for Web which is taking the same route. Call it ORTC instead of WebRTC - they are one and the same as they are set to merge anyway.

Apple? Will they place FaceTime on top of WebRTC? I see no incentive there whatsoever.

-

Can Cisco change this? Rowan Trollope broke the news titled “Cisco and Apple Announce New Features” that WebEx and Cisco Spark now work on Safari - no download needed. I’ll translate it for you by adding the missing keyword here - WebRTC. Cisco is using WebRTC to do that. And since their stack is built atop H.264, they got that working on Safari.

Cisco and Apple here is interesting. While Cisco mentions this in the headline as if these new features were done jointly, Apple isn’t really acknowledging it. There’s no quote from an Apple representative, and at the same time, Cisco isn’t mentioned in the WebKit announcement - TokBox and BlueJeans are.

One wonders.

-

Back to FaceTime. Which is a 1:1 video chat service.

And the fact that many look into group video chat and other multiparty video configurations.

Will Apple care enough to support it well in WebRTC? Will it move from Plan B to Unified Plan? Will it care about simulcast? Will it invest in SVC? Will it listen and work with Cisco on its multiparty needs for the benefit of us all?

Older Devices

Apple will not be upgrading iPhone 5 devices to iOS 11. That’s a 5 years old device.

In many requirement documents I see a request to support iPhone 4.

Will this bump the general audience out there to focus on iPhone 6 and upwards, seeing what Apple is doing as well? Will this mean vendors will need to port WebRTC on their own to support older devices?

Time will tell, but I think that switching to iPhone 6 and above and focusing there makes sense.

Chrome/Firefox support on iOS

Here’s a question for you.

If you use Chrome or Firefox on iOS. And open a URL to a site using WebRTC. Will that work?

Here’s the catch.

The reason there was no real browser for iOS that supported iOS until today? Apple mandates WebKit as the rendering engine on any browser app that comes to its AppStore.

Now that WebKit is getting official WebRTC support - will Chrome and Firefox add WebRTC support to their iOS browsers?

And if they do - you’ll be getting the Apple restrictions. I can just see the WebRTC developer teams at Google and Mozilla cringing at this thought.

This can get really interesting if and when Apple decides to add HEVC support to WebRTC in its WebKit implementation of iOS. You’ll get Chrome on iOS with H.264 and HEVC and Chrome everywhere else with H.264, VP8 and VP9.

Fun times.

What Should Developers Do?

Here’s what you’ve been waiting for. The question I’ve been asked multiple times already:

Do I need to build an app? Should I wait?

The suggest at the moment is wait. Question is for what and until when exactly.

If you are looking for a closed app and planning on developing native, then go with whatever worked for you until today. This news item just isn’t for you.

If you are looking for browser support on iOS, then go with Safari and plan on enabling H.264 video codec in your service. Don’t wait up for VP8.

If you want something that will be cross platform app development using HTML5, then wait. Webview WebRTC support in iOS is still an unknown. If it gets there in the coming months then waiting a few more minutes probably won’t make a real difference for you anyway.

My Updated Cheat Sheet

As it is, this change with Safari, iOS and macOS required some necessary updates in my resources.

First to update is the WebRTC Device Cheat sheet. You can find the updated one in the same download page.

One last thing -

Join my and Philipp Hancke for Virtual Coffee

I planned for a different Virtual Coffee session this month. One about developer tools. It got bumped into July.

The one in June will cover the iOS announcement and its ramifications. My guest this time will be Philipp Hancke.