Matt Fleckenstein on Mixed Reality and New Microsoft Mesh

As a member of the AR ecosystem, you’re familiar with Microsoft HoloLens, of course. But are you aware of Microsoft’s AR strategy beyond its successful head-mounted display? To get answers, we spoke to Matt Fleckenstein, Head of Strategy, Commercial and Consumer Mixed Reality for the IT giant, which recently joined the AREA as a Sponsor member. A tech veteran with a rich and varied background, Matt gave us a fresh look at the future of AR/VR/MR at Microsoft.

AREA: Matt, welcome to the AREA! You’ve been in your new position at Microsoft for a few months now. Tell us about your role and the scope of your work.

Fleckenstein: I got recruited back to Microsoft two years ago, and I’ve spent most of my time driving our Mixed Reality business. I was on the product marketing team for the majority of that time, where I ran our go-to-market for our Mixed Reality business, and more recently I moved over to our engineering team to drive our Mixed Reality strategy and road map for our products and services.. We’re starting to look at what, if anything, we’re going to do beyond the VR stuff we have in market today in the consumer world. The commercial part is probably more apropos to what we’re talking about here.

AREA: You’re looking at both the commercial and consumer sides of things. Is there a lot of overlap between the two or are they pretty distinct?

Fleckenstein: It’s a great question. They’re fairly distinct today, but my point of view is that they’ll look more similar down the road. What I mean by that is, when you think about the world we’re in today, where we’re all forced to interact in much more remote manner. And whether I’m interacting with other people for work purposes or in my personal life, over time those things will look more and more alike.

A good example of that is our recent announcement of Microsoft Mesh. Microsoft Mesh is an enabling platform to usher in the era of collaborative computing, enabling users to work, play, and learn together remotely, as if they were there in person. Using any MR-enabled device (such as HoloLens, Quest, or AR-enabled phones), users can holoport themselves or their avatars into any location to enable rich, immersive shared experiences. This has broad applicability across both commercial and consumer businesses from hosting virtual conferences to remote meetings to rock concerts. When you think about it in that context, even though the content is different, the ability to connect with other humans and collaborate is a human need across our work and personal lives.

AREA: I would imagine, too, that you’re going to learn different things from the commercial and consumer sides that could be transferable between the two.

Fleckenstein: Yeah, I think that’s true. As the folks in the AREA well know, a lot of the use cases for enterprise Augmented, Mixed, and Virtual Reality today are in training. And some of that training is teaching people hard skills – for example, how to assemble a Rolls-Royce engine for an Airbus airplane. But others are more about teaching soft skills, whether it’s customer service skills or sales training and skills training. When you start to get into that world, it’s all about human connection and emotion, as well. So, while I do think they are different today, they will also definitely feed each other. There is more overlap probably than what may be apparent at first blush.

AREA: Can you give us a sense of the scope of Microsoft’s Mixed Reality strategy?

Fleckenstein: The biggest challenge I had when I was on the marketing side was overcoming the perception that we were about HoloLens and that was it. Certainly, that’s a big part of what we do. But we’ve tried to holistically build out a full Mixed Reality stack so that you can have all the tools and services running in Azure that you need in conjunction with 3D content platforms like Unity and Unreal. So, that you can build the applications and manage the applications that you need in the enterprise space – whether that’s running on a HoloLens device or a mobile device. You know, we’ve built a rich set of mixed reality Azure services and support that development with deep integration with those 3D content platforms. So, we’ve got kind of that PaaS layer that really can be helpful.

We also have our first-party SaaS applications. We have two applications in our Microsoft Dynamics 365 group: Remote Assist and Guides. Remote Assist is all about the use case for bringing in a remote expert to help you with whatever you’re doing. That could be a field service agent who’s out trying to repair a complex piece of machinery and needs an additional set of eyes and ears and hands to help do what they’re doing. Or it could be a medical professional who wants to patch in an expert to get a second opinion on something. Guides is all about the step-by-step guidance to help you perform a task. Go back to that Rolls Royce engine for an airplane, right? Teaching a new employee how to assemble or how to repair one of those – leveraging the power of overlaying 3D holographic images on top of what you’re doing to walk you through exactly what you need to do is one of the use cases that we see today. So, we’ve got our own first-party set of applications that run on HoloLens but also run on other endpoints like mobile devices and tablets.

And then we’ve got Microsoft Mesh and a variety of mixed reality PaaS services that have enabled a robust ecosystem of about 175 different enterprise ISV’s who have built applications to solve different use cases in different industries, some leveraging HoloLens and some leveraging other endpoints. The industries served include health care, manufacturing, architecture, construction, and engineering.

Then of course we’ve got the AltspaceVR service where people can gather in VR and do everything from rock concerts to conferences to virtual meetings. It’s a pretty robust stack when you look at what we’re kind of trying to build out, end to end. Ultimately, our focus is to leverage the power of Augmented, Virtual, and Mixed Reality to enhance the next generation of experiences across consumer and commercial worlds.

AREA: What’s at the top of your to-do list for 2021?

Fleckenstein: We’re doubling down on how we can, at scale, light up the next generation of collaboration as evidenced by our launch of Microsoft Mesh. A lot of the work that’s been done in enterprises today, a lot of the success that folks have had with Mixed Reality, with HoloLens, has been helping individuals complete a job. I think the opportunity exists for much more. This dialogue we’re having remotely through a 2D medium would be so much richer if you and I were sitting down having a cup of coffee in front of the fireplace having a chat. So our focus is on how we light up the next generation of collaboration, whether it’s me talking to my parents back in Punxsutawney, PA to get an update on whether Phil saw his shadow or not, or having a work meeting with a bunch of people. How can we best transcend time and space and create that next generation of collaboration?

AREA: What do you see as the most important barriers that need to come down to open up all these opportunities?

Fleckenstein: There’s a whole set of practical obstacles in the enterprise that we need to keep working through. Those include everything from making it easier for IT departments to manage a suite of different devices to enable Mixed Reality experiences in their enterprises. That’s everything from security to device management to dealing with authentication and identity. There’s more work to do to make all of that easier so that we can move from companies deploying dozens or hundreds of these devices to thousands to 10,000’s of these devices.

There are certainly companies who are leading the way on that, like Toyota, for example, which is putting HoloLens devices in every one of their dealerships in their service centers so that they can offer next-generation customer experiences. That’s going to happen in the tens of thousands, but that’s rare still today. So, the main obstacles are the physical IT deployment and rollout tasks that need more maturity.

AREA: Right. The less glamorous stuff.

Fleckenstein: Yeah, but it’s absolutely necessary. We saw it with mobile devices when people started bringing their iPhones to work. You have to do that foundational work if you want to operate it at scale. The other key is change management. New technologies are always tough to introduce. How do you bring along the people and the processes to support the evolution that’s happening? How do we manage the transition from mobile devices and computer screens to head-mounted displays and ultimately glasses?

The expanded adoption of Mixed Reality will also require us to defy physics by making the devices smaller and more lightweight while making the displays and the field of view better and bigger and improving battery life all at the same time. I do think smaller form factors that lower costs are really going to be imperative for us to really unlock the full potential of the technology. We just need to keep making the products lighter and better and bringing down that cost.

AREA: What does Microsoft expect to get out of being a member of the AREA?

Fleckenstein: One of the things that was most attractive, and one of the reasons why we joined, was to get access to a lot of the research and tools the AREA has put together. There’s just nothing else like it out there – whether it’s an incredibly robust worksheet to help you build out the business case for your organization, or the research the AREA has conducted to understand what people are really doing with the technology. What are the challenges that they’re facing? What are the requirements that aren’t being met? There’s tons of research out there about this space in general, but it’s fairly high level and it’s fairly broad. One of the biggest things the AREA has going for it is the rigor and depth of the research. It’s been fantastic.