Mozilla Takes Aim at the AR Browser

Mozilla is the not-for-profit behind the popular web browser, Firefox. The company works to ensure it stays open by building products, technologies and programs that put people in control of their online lives. In April of this year, Mozilla officially announced Firefox Reality, a browser designed specifically for VR and AR headsets. Firefox Reality currently runs in developer mode on Google’s Daydream and Samsung’s and Ocular’s Gear VR, with more devices on the way. We spoke recently with Lars Bergstrom, Director of Engineering, Mixed Reality at Mozilla Research, to learn more about Mozilla’s plans.

AREA: Tell us about how Mozilla decided to get into augmented reality.

Bergstrom: As a nonprofit, mission-driven organization, Mozilla believes in protecting the user online. We want everyone to be able to get online, use information on the Internet and have it delivered to them in a safe and private manner. We had been experimenting with VR and AR headsets for a while and saw a couple of years ago that the technology was finally getting to a point where it’s ready for wide scale adoption in enterprises and very soon, consumers. And so, in order to get to a place where we could ensure that the web was safe, private and relevant for consumers, we knew that we needed to start investing in building browsers and web-based technologies for these VR and AR headsets so that, by the time people were ready to buy them, the web was already there.

AREA: What’s different about Firefox for AR compared to other headset-enabled browsers?

Bergstrom: The biggest thing we’re trying to do is to partner and build a really great immersive experience for each headset, customized for the hardware. Each one has its own sweet spot, field of view, and ergonomics. What gestures does it support? How long can people use it and be comfortable? We’ve tried to work with each of the headsets, either with the manufacturers directly or ourselves, to figure out how to build a browser that’s customized to work well on that hardware, is comfortable, and offers something that users want to be in all the time. So, one of the biggest things that distinguishes us is that we’re working directly with the hardware vendors.

AREA: What are some of the fine points of browser design when you’re talking about the augmented reality environment?

Bergstrom: The web as a platform didn’t do a lot with computer vision. The camera is only available to the browser through generic media APIs, but when we transition to augmented reality, everyone using these devices expects to have access to the specialized hardware APIs available on those devices. They expect to have some sort of object recognition, they expect to be able to find QR codes, and they expect reasoning about the environment, whether that’s object detection or surface detection. So one of the big things that we’ve had to do in the browser was work with the hardware vendors and other people to determine how we were going to expose this information, up to web pages and do it in a safe way? How are we going to ask users for privacy? Because right now, if you build a Unity app, you just get access to all of these APIs and then there’s some assumption of trust. You install the application and it doesn’t really prompt you again. However, with the web, we’re talking about potentially untrusted content; you go to some website that’s selling a poster and they ask you if you want to put the poster on the wall. We need to be very careful for the user. How are we asking them for permission to display that in the real world and how much information is actually being transmitted back to that website? Because the expectation of trust is very different.

AREA: It sounds as if you’re trying to be very responsible about safety and privacy.

Bergstrom: We have to. We are a nonprofit and we’re mission-driven so that is our measure of success – getting people to safely and privately use the Internet on these devices.

AREA: You announced Firefox Reality in April. What’s the roadmap and what milestones can we look forward to?

Bergstrom: We’re focusing on the standalone device experience right now. We think mobile phone AR is really great, as is mobile phone VR, for getting developers experimenting with the hardware. But it’s hard to see people integrating it with their daily lives until it’s in a standalone headset. So, we’ve been focusing on all of the unique requirements there. For example, how do you load a URL when you don’t have an onscreen keyboard to fall back on? Our latest milestone is the release of versions of both the VR and AR browsers in application stores for several standalone headsets.

AREA: And after that?

Bergstrom: The big thing we’re working on is going beyond stores and actually getting it installed on devices for distribution with the headsets because that’s really where we want to be. We want to make sure that all of these headsets that ship have a good browser on them for doing web-based virtual and augmented reality.

AREA: What does Mozilla hope to get out of its membership in the AREA?

Bergstrom: One of the things that I really like about the AREA is that it brings together a lot of very experienced large organizations that have been running pilots and programs with augmented reality for many years. These are organizations we normally wouldn’t have contact with, as a nonprofit organization that’s mainly consumer focused. The AREA bridges that gap to help us connect to enterprises and industry in the way that we haven’t traditionally had. I am always surprised at how much I learn every time I talk to members of the AREA.

AREA: What kind of response have you had from the marketplace about the Firefox Reality announcement?

Bergstrom: We wanted to reach out to more hardware vendors and strongly signal to the market that we’re bringing a browser to AR and VR headsets and they should reach out to us. In our view, there’s no value in being exclusive to one device. Most of the value comes when we are on every device. Then people can build a web app and they get the experience to work across all of them. We’ve gotten a lot feedback. Immediately after we went live, a number of hardware vendors reached out to us. They’ve tried it on their devices and they’d like us to tune it for their hardware. We’ve also engaged developers to begin experimenting with it. Their feedback has been, “Okay, when are you going to have this installed on devices by default so I can ship my application?”

AREA: What’s the biggest question developers have?

Bergstrom: How is the performance relative to a native solution? In some ways, the web is not as mature a platform as Unity, in particular in terms of access to the raw graphics compute hardware. So we’re working to help developers get answers to those questions.