With a heritage dating back more than a century, Rockwell Automation has long been at the forefront of industrial automation. Today, the $7 billion company offers a broad array of solutions and services that bring “The Connected Enterprise®” to life, including automation systems, integrated control systems, and factory software. We spoke with Rockwell’s Andrew Ellis shortly after the company joined the AREA as a Sponsor member.

AREA: Everybody knows Rockwell Automation – it’s a huge company with lots of different businesses in industrial automation. Could give us a perspective on your specific organization within Rockwell?

Ellis: Sure. I’m the director of our global tech consultants within a new office called the Digital Technical Office. Our tech consultants in the field help our customers align their digital transformation requirements with our capabilities. I also oversee a group of experts in different disciplines, including XR, Extended Reality. Again, our role is really working with the customers to understand their challenges and then align our portfolio to help them along on that journey.

AREA: Did the interest in XR come about because customers were asking about it, or was it because you looked into the future and said this is something we need to get into?

Ellis: When I first looked at AR several years ago, I was a bit skeptical, but it didn’t take long to change my point of view completely. Today, I don’t know how, in the near future, any customer will be able to move forward with a modern manufacturing facility without AR. It’s now similar to how it was in my early days of automation and IT/OT; there’s a lot of education going on with our customers to make them aware of the use cases, the problems AR solves, and the real benefits in an industrial landscape. For me personally, I’ve done a complete 180-degree turn around from “not sure” to “wow, you’ve got to go all-in on AR.” From our vustomer’s point of view, more and more they are asking for it, there’s no doubt. Almost on every opportunity, there are two things that come up – machine learning and Augmented Reality. How can they take advantage of it? It’s an extremely hot topic.

And of course, Rockwell’s interest and activities in AR accelerated considerably three years ago, when we made a $1 billion investment in PTC to partner with PTC and make it part of our InnovationSuite. Along with that came not only ThingWorx, their IoT platform, but Vuforia, their Augmented Reality platform.

AREA: What are the top industrial applications for AR that you’re talking to customers about?

Ellis: Number one is remote support. For us, that’s the application of Vuforia Chalk. The power of AR, using the camera on your phone to actually see it and then mark it up is huge, and to me it’s a no brainer. That’s a tool that needs to be in any field service person’s bag. Another is any kind of work instruction, whether simple or complex, where you want to have consistent methodology, and you want to get away from having to read 20 pages of a manual. The ramp-up time to understand how to assemble something, or for maintenance, to disassemble and repair something, is greatly shortened. That time savings is very valuable for customers.

A third popular use case is when you want to get a group of unskilled people up to speed quickly on a complex operation. They can use AR to walk them through the process – here’s the tools you need, here’s the safety instruction before you begin, here is the work instruction you need to follow – all manufacturers need to have this capability within their facility.

AREA: So the advantages of AR in all of those use cases are greater speed and efficiency, lower cost, and greater agility, right?

Ellis: Right. Even though we say we digitize our manuals, we’re simply making them digital as a file. The consumption still requires you to sit there and read something, whereas having it overlaid in front of you is just a totally different and better way of visualizing work instruction. The younger workers coming out of university and college are used to this. This is the world they want to live in, so we’ve got to provide the tools that are appropriate to them.

AREA: Have you seen the level of knowledge among customers increase a lot in recent years, or is there still a lot of education that has to go into the process?

Ellis: For our customers, there’s been a huge increase in awareness of what AR is and how it’s used. They’re now more interested to know what other customers have done. Nobody wants to be that first adopter. They want to be somewhere near the middle, and not the laggards. I think we’re still on that upward curve. It definitely hasn’t become standard in every processing plant or manufacturing facility, but there’s lots of research going on regarding how to apply it.

They want to know what other companies are getting for a return on investment. How did they deploy? What kind of infrastructure did they need? It is something that you need to have a solid wireless infrastructure available in your facility if you plan to use it for certain things, or a 5G network, whatever you want to use, but it makes the infrastructure really a prerequisite.

AREA: When you’re talking to prospects, and they say give me your best case study that you can share publicly, what do you tell them?

Ellis: Because we’re leveraging PTC product, a lot of our case studies are from PTC, because they’ve been doing this for a lot longer than us. But we also talk about our own internal case studies, because we’re a manufacturer as well, we’ve got 20-plus manufacturing facilities globally, and using AR with our equipment makes a lot of sense. Currently we’re working on an application where you can use AR to look at our MCC motor control centers, or our variable speed drives, or variable frequency drives, and start to get IoT data out of them. And not only the data, but start to get the predictions, obviously the predictions are coming from somewhere else, but now they’re showing up in AR.

Or I can use it as virtual safety boundaries. If I’m using the HoloLens, and I’ve got spatial awareness, as I move into an area that may be considered dangerous, it’s fenced off virtually. We’re seeing a lot of benefit applying AR experiences to our own products to help our customers, as well as in our own manufacturing facilities when we’re doing virtual commissioning of equipment. So we can use Vuforia Chalk to look at an assembly and share it with a customer in the markup. That kind of virtual commissioning has been fantastic. So we’ve had a number of use cases where we’re using it ourselves in our own facilities.

AREA: Is there a vision for AR at Rockwell Automation, or is it considered just another tool in the toolbox you’re offering customers?

Ellis: I think it’s becoming part of our vision, a fundamental pillar in what we will provide to our customers. We can see a day when someone puts on either a 2D or 3D device, looks at their equipment or facility, and starts to visualize where the trouble spots are, what product is running through the line at this point in time, who is on the line working on it, what are the orders that are jammed up. You’ll be able to really visualize it so you don’t have to go into a traditional control room or open up your laptop to take a look at it.

We haven’t tapped all of it, but there’s so much interest in using AR in almost all of our products, from hardware in maintenance and safety, to assembly, to how to use, to troubleshooting, and then on the IoT side, consuming IoT data, or work instructions. It’s used everywhere, so I think it’s going to be a standard platform that you’ll see from Rockwell from now on.

AREA: Finally, could you tell us your reasons for joining the AREA?

Ellis: Our customers – whether it’s manufacturers or processing facilities – get a lot of their information from independent organizations like the AREA that are solely focused on sharing best practices standards and information gleaned from other users, and making it available to the wider ecosystem. That’s what the AREA does, and we wanted to be part of it – to contribute to it, sharing what we’re seeing as a manufacturer and what our customers are seeing, but also to gain access to the information the AREA is putting together. Your calculators are fantastic. We can point our customers to AREA literature that’s vendor-neutral – much like when I’m buying something, I want to look at Consumer Reports or another source that has no vendor bias. When you have an organization like the AREA, where you pull in vendors and users, and create material that we can consume, but more importantly share with our customers – to me that’s invaluable.

New AREA member EXO Insights has carved a successful niche in the enterprise AR market as a source of biometric VR/AR solutions that transform and build on existing industrial safety and training standards to move organizations past traditional training methods. Based in Waterloo, Ontario, EXO Insights’ suite of sensors, integrated with VR/AR hardware, collects precise behavioral data generated during sector-specific designed simulations.

The EXO Analytics engine then consolidates trainees’ knowledge and behavioral data to provide organizations with actionable insights to improve industrial safety and productivity. We spoke to the company’s CEO, Fernando Muniz-Simas to learn more about EXO Insights and its unique AR-based solutions.

AREA: Welcome to the AREA, Fernando. Tell us how EXO Insights started and how you got to where you are today.

Muniz-Simas: The company started unofficially about 15 years ago, pre-Augmented Reality. We began in the Marketing area, doing research for big food conglomerates like Nestlé and Unilever. We were leveraging Virtual Reality as a way of capturing behavioral information for market research through eyetracking. We created simulated supermarket aisles and asked people to reenact their shopping trips. It was valuable information to our clients because it was unbiased; we simply measured behavior.

While the research was effective, it was very costly. So we began looking at applying our ideas to industrial training applications in the mining industry and our business took off. We established our company here in Canada in 2016 and started working in the energy and defense sectors, always in a mission-critical, high-risk environment.

AREA: Are most of your customers in Canada?

Muniz-Simas: Yes. We also have one customer in Europe and we are working to expand our business across North America.

AREA: Is your strategy to stay in the training space?

Muniz-Simas: Yes. We are also strongly focused on research. We’re very close to an important research university, the University of Waterloo, and this whole area is a great center of innovation and entrepreneurship. There are two components to our work: using sensors that measure behavior; and making that information meaningful. We could be measuring heart rate and stress, for example, but is that information meaningful to the task at hand? Research is fundamental for us.

AREA: Who’s a typical EXO Insights customer and what are they doing with your solutions?

Muniz-Simas: A good example is nuclear power generation. They are tremendously focused on safety, for a very good reason. It’s a very good match for our simulation capabilities, because we’re not really focused on training per se, but on the finer points of how behavior affects the outcomes of your training or how you perform your work. There are two ways to improve skills: classroom training and human performance – how you improve the way you do your work. It’s not necessarily related to specific skills. For example, the nuclear industry trains some of its operators in flight simulators. It’s as advanced as you get. It’s high-risk, mission-critical work.

EXO Insights helps them improve human performance by delivers training that is as close to reality as possible. Leveraging state-of-the-art VR, AR, and biometric technologies, we create a safe and hyper-real training experience that allows workers to rehearse critical tasks and put their learning to the test in a safe, simulated workplace. Simulation allows workers and organizations to safely fail, so they can learn from those failures and ensure they are not repeated in real life.

AREA: Tell us why you joined the AREA and what you hope to get out of your membership.

Muniz-Simas: Since coming to Canada four years ago, we’ve essentially become a startup again, so we’re looking for ways to learn how things are done here. I’ve always had the AREA close to my heart. The organization has always had open arms and has freely shared some really great information – for example, the ROI Calculator. There’s a lot of great thinking behind that. We saw a lot of value in the AREA and a lot of correctly-focused initiatives that will strengthen the AR ecosystem. It’s a very solid organization with some very important members. It’s something we felt we had to participate in and contribute to.

As a member of the AR ecosystem, you’re familiar with Microsoft HoloLens, of course. But are you aware of Microsoft’s AR strategy beyond its successful head-mounted display? To get answers, we spoke to Matt Fleckenstein, Head of Strategy, Commercial and Consumer Mixed Reality for the IT giant, which recently joined the AREA as a Sponsor member. A tech veteran with a rich and varied background, Matt gave us a fresh look at the future of AR/VR/MR at Microsoft.

AREA: Matt, welcome to the AREA! You’ve been in your new position at Microsoft for a few months now. Tell us about your role and the scope of your work.

Fleckenstein: I got recruited back to Microsoft two years ago, and I’ve spent most of my time driving our Mixed Reality business. I was on the product marketing team for the majority of that time, where I ran our go-to-market for our Mixed Reality business, and more recently I moved over to our engineering team to drive our Mixed Reality strategy and road map for our products and services.. We’re starting to look at what, if anything, we’re going to do beyond the VR stuff we have in market today in the consumer world. The commercial part is probably more apropos to what we’re talking about here.

AREA: You’re looking at both the commercial and consumer sides of things. Is there a lot of overlap between the two or are they pretty distinct?

Fleckenstein: It’s a great question. They’re fairly distinct today, but my point of view is that they’ll look more similar down the road. What I mean by that is, when you think about the world we’re in today, where we’re all forced to interact in much more remote manner. And whether I’m interacting with other people for work purposes or in my personal life, over time those things will look more and more alike.

A good example of that is our recent announcement of Microsoft Mesh. Microsoft Mesh is an enabling platform to usher in the era of collaborative computing, enabling users to work, play, and learn together remotely, as if they were there in person. Using any MR-enabled device (such as HoloLens, Quest, or AR-enabled phones), users can holoport themselves or their avatars into any location to enable rich, immersive shared experiences. This has broad applicability across both commercial and consumer businesses from hosting virtual conferences to remote meetings to rock concerts. When you think about it in that context, even though the content is different, the ability to connect with other humans and collaborate is a human need across our work and personal lives.

AREA: I would imagine, too, that you’re going to learn different things from the commercial and consumer sides that could be transferable between the two.

Fleckenstein: Yeah, I think that’s true. As the folks in the AREA well know, a lot of the use cases for enterprise Augmented, Mixed, and Virtual Reality today are in training. And some of that training is teaching people hard skills – for example, how to assemble a Rolls-Royce engine for an Airbus airplane. But others are more about teaching soft skills, whether it’s customer service skills or sales training and skills training. When you start to get into that world, it’s all about human connection and emotion, as well. So, while I do think they are different today, they will also definitely feed each other. There is more overlap probably than what may be apparent at first blush.

AREA: Can you give us a sense of the scope of Microsoft’s Mixed Reality strategy?

Fleckenstein: The biggest challenge I had when I was on the marketing side was overcoming the perception that we were about HoloLens and that was it. Certainly, that’s a big part of what we do. But we’ve tried to holistically build out a full Mixed Reality stack so that you can have all the tools and services running in Azure that you need in conjunction with 3D content platforms like Unity and Unreal. So, that you can build the applications and manage the applications that you need in the enterprise space – whether that’s running on a HoloLens device or a mobile device. You know, we’ve built a rich set of mixed reality Azure services and support that development with deep integration with those 3D content platforms. So, we’ve got kind of that PaaS layer that really can be helpful.

We also have our first-party SaaS applications. We have two applications in our Microsoft Dynamics 365 group: Remote Assist and Guides. Remote Assist is all about the use case for bringing in a remote expert to help you with whatever you’re doing. That could be a field service agent who’s out trying to repair a complex piece of machinery and needs an additional set of eyes and ears and hands to help do what they’re doing. Or it could be a medical professional who wants to patch in an expert to get a second opinion on something. Guides is all about the step-by-step guidance to help you perform a task. Go back to that Rolls Royce engine for an airplane, right? Teaching a new employee how to assemble or how to repair one of those – leveraging the power of overlaying 3D holographic images on top of what you’re doing to walk you through exactly what you need to do is one of the use cases that we see today. So, we’ve got our own first-party set of applications that run on HoloLens but also run on other endpoints like mobile devices and tablets.

And then we’ve got Microsoft Mesh and a variety of mixed reality PaaS services that have enabled a robust ecosystem of about 175 different enterprise ISV’s who have built applications to solve different use cases in different industries, some leveraging HoloLens and some leveraging other endpoints. The industries served include health care, manufacturing, architecture, construction, and engineering.

Then of course we’ve got the AltspaceVR service where people can gather in VR and do everything from rock concerts to conferences to virtual meetings. It’s a pretty robust stack when you look at what we’re kind of trying to build out, end to end. Ultimately, our focus is to leverage the power of Augmented, Virtual, and Mixed Reality to enhance the next generation of experiences across consumer and commercial worlds.

AREA: What’s at the top of your to-do list for 2021?

Fleckenstein: We’re doubling down on how we can, at scale, light up the next generation of collaboration as evidenced by our launch of Microsoft Mesh. A lot of the work that’s been done in enterprises today, a lot of the success that folks have had with Mixed Reality, with HoloLens, has been helping individuals complete a job. I think the opportunity exists for much more. This dialogue we’re having remotely through a 2D medium would be so much richer if you and I were sitting down having a cup of coffee in front of the fireplace having a chat. So our focus is on how we light up the next generation of collaboration, whether it’s me talking to my parents back in Punxsutawney, PA to get an update on whether Phil saw his shadow or not, or having a work meeting with a bunch of people. How can we best transcend time and space and create that next generation of collaboration?

AREA: What do you see as the most important barriers that need to come down to open up all these opportunities?

Fleckenstein: There’s a whole set of practical obstacles in the enterprise that we need to keep working through. Those include everything from making it easier for IT departments to manage a suite of different devices to enable Mixed Reality experiences in their enterprises. That’s everything from security to device management to dealing with authentication and identity. There’s more work to do to make all of that easier so that we can move from companies deploying dozens or hundreds of these devices to thousands to 10,000’s of these devices.

There are certainly companies who are leading the way on that, like Toyota, for example, which is putting HoloLens devices in every one of their dealerships in their service centers so that they can offer next-generation customer experiences. That’s going to happen in the tens of thousands, but that’s rare still today. So, the main obstacles are the physical IT deployment and rollout tasks that need more maturity.

AREA: Right. The less glamorous stuff.

Fleckenstein: Yeah, but it’s absolutely necessary. We saw it with mobile devices when people started bringing their iPhones to work. You have to do that foundational work if you want to operate it at scale. The other key is change management. New technologies are always tough to introduce. How do you bring along the people and the processes to support the evolution that’s happening? How do we manage the transition from mobile devices and computer screens to head-mounted displays and ultimately glasses?

The expanded adoption of Mixed Reality will also require us to defy physics by making the devices smaller and more lightweight while making the displays and the field of view better and bigger and improving battery life all at the same time. I do think smaller form factors that lower costs are really going to be imperative for us to really unlock the full potential of the technology. We just need to keep making the products lighter and better and bringing down that cost.

AREA: What does Microsoft expect to get out of being a member of the AREA?

Fleckenstein: One of the things that was most attractive, and one of the reasons why we joined, was to get access to a lot of the research and tools the AREA has put together. There’s just nothing else like it out there – whether it’s an incredibly robust worksheet to help you build out the business case for your organization, or the research the AREA has conducted to understand what people are really doing with the technology. What are the challenges that they’re facing? What are the requirements that aren’t being met? There’s tons of research out there about this space in general, but it’s fairly high level and it’s fairly broad. One of the biggest things the AREA has going for it is the rigor and depth of the research. It’s been fantastic.

We all like to consume our content in different ways. For many of us, listening and watching can be easier and more digestible than reading. That’s why we’re pleased to offer all of the AREA’s public webinars and thought leader interviews on our YouTube channel. You’ll find years worth of great content devoted to Augmented Reality in the Enterprise there. 

Better yet, why not subscribe to keep up to date on everything connected with AR for the Enterprise?

«

Prev

1

/

12

Next

»

AREA Founder Christine Perey Named to XR Hall of Fame

Introduction to AREA RoI calculator and Use Cases

AREA Trends for 2024: New Interaction Modalities for AR (Part 2 of 2)

Fireside Chat: Inside the AR Center (ARC) at Oakland University.

Top AR Trends for 2024: New Interaction Modalities for AR (Part 1 of 2)

Challenges of Delivering WebXR in the Enterprise

Top AR Trends for 2024: AR Ecosystem Standards - Jonathan Hale, Wonderland Engine

How Voice-Assisted Apps Ensure Safety of Frontline Workers

Voice Activated Apps: The Key to Productivity and Safety for Frontline Workers

Top AR Trends for 2024: Hardware - Brian Hamilton and Nima Shams, DigiLens

The AREA is Ten Years Old!

AR for Manufacturing: How to Select the Most Relevant Use Case

AR for Manufacturing, How to?

What to Expect at ARVR Innovate 2024: May 8, Dublin

Fireside Chat: How the Integration of 5G and AR Impacts Business Efficiency

«

Prev

1

/

12

Next

»

Stereoscape, a new AREA member, was founded in 2009 in Helsinki, Finland by a team of creative and technology experts specializing in stereoscopic 3D for films. Today the company is focused on using interactive 3D and XR technologies to enable people to work smarter and learn faster. We spoke with the Stereoscape team recently to learn more.

AREA: Tell us about how Stereoscape got into AR.

Stereoscape: We started 10 years ago with stereoscopic 3D conversions and live action S3D shoots for film, fashion and advertising clients, after which we moved to 3D holographic solutions for marketing and events. Over time, our focus turned to working with immersive and interactive technologies, and we now have five years’ experience in XR-based projects for enterprise clients. Throughout our 10-year journey we’ve been passionate about how the creative application of new technologies can deepen people’s experiences, whether at work or leisure.

AREA: Are you focusing primarily on the European market?

Stereoscape: At the moment, we are mainly working with large Finnish multinationals in the industrial B2B segment. Some of our clients build mechanical engineering products, while others, like Nokia, are providing technology for 5G networks and Industry 4.0. Our key clients are based in Finland and the Nordics, with our solutions often deployed internationally.

AREA: How would you describe the state of AR adoption in Finland?

Stereoscape: Finland is still very much at the emerging stages of adopting AR, with some great success stories in validated solutions, while others are still learning about the potential for AR as a driver of business value. Because Finland has a strong tech infrastructure and tech startup culture, there is generally a good appreciation for technology-driven innovation, and there’s a nice AR ecosystem forming locally.

AREA: What would you say distinguishes Stereoscape from its competitors?

Stereoscape: With the rapid pace of innovation in XR, we remain technology-agnostic. We are focused on the user experience – putting technology in the service of people. We are passionate about making people’s working lives easier by presenting information in an engaging, interactive and effective way with the help of XR. Our expert team has accumulated a wealth of experience in the creative applications of new technologies. We’ve also invested in the productization of our solutions and services to make it easier for our enterprise clients to adopt these new technologies.

AREA: On your website, you talk about “Smart Product Communication” as being the underlying theme of your work. What do you mean by that?

Stereoscape: We first coined the concept some five years ago to depict the new ways of communicating complex product information made possible by interactive 3D and XR – moving from static to dynamic, from flat to spatial, from consumption to exploration, from explanation to experience. We were then mainly working with sales and marketing solutions and have since extended our offering to other parts of the enterprise value chain. But helping people comprehend complex information is still at the heart of everything we do, whether in sales, learning or maintenance. With AR, and other XR, we have so many exciting tools available to make digital information more alive, accessible, responsive and enjoyable. I think our slogan – Experience information – nicely captures our mission of enabling people to experience information in entirely new ways.

AREA: Your fellow AREA member RE’FLEKT is one of your partners. Do you do a lot of work in collaboration with partners?

Stereoscape: We work with a broad international network of technology and other expert partners on all our projects, including developers and other subcontractors. In-house, we focus on client consultation and the design of the solution, then collaborate closely with our partner network on the implementation phase to create an end-to-end solution.

AREA: What do you hope to gain by joining AREA?

Stereoscape: We are keen to connect with other companies and our peers who are passionate about AR to exchange experiences, learn, and collaborate. AR, and XR in general, requires a wide range of experts to join forces to bring about the digital transformation made possible by new technologies.

AREA: What can we look forward to from Stereoscape in the next couple of years?

Stereoscape: Since Covid-19 broke out, we have been busy creating new solutions for online use, to help companies engage with their customers or onboard and train employees despite the meeting and travel restrictions. Looking beyond the current crisis, we firmly believe that the pace of AR innovation and adoption will only keep accelerating. We are excited by the opportunities this will bring to our clients and all those working within the field of AR. 

Whether or not you agree with the old saying that “cleanliness is next to godliness,” in the COVID-19 age, cleanliness is paramount when it comes to preventing infection. The U.S. National Institutes of Health report that SARS-CoV-2, the virus that causes COVID-19, can be detected in aerosols for up to three hours and on plastic and stainless steel surfaces for up to three days.

That’s why it’s so vital to regularly – and thoroughly – clean all AR devices, particularly for organizations where headsets and handheld devices are shared among multiple users. We consulted with the AREA Safety Committee to learn how best to handle headset cleaning.

While no official industry best practices have been established for coronavirus cleaning (and there are no guarantees that your cleaning regimen will prevent transmission of the disease), the AREA would like to share these recommendations. They begin with separate cleaning materials and processes for each AR headset component, as follows:

Optics 

Lenses and cameras can be damaged by abrasive cloths or cleaners. The Safety Committee’s advice is to unplug the camera from its power source, then wipe clean with a microfiber cloth. Lenses get the same treatment, with the microfiber cloth applied in a circular motion, beginning at the center of the lens and moving outward. 

Cushions 

To clean headset cushions, first remove the cushion from the headset. Take a soft cloth dampened with warm (40°C/104°F) water, gently wipe, and allow to dry at room temperature. 

Headset Plastics 

Hard surfaces, such as headset plastics, are where the coronavirus is likeliest to survive. The Committee recommends isolating the component, wiping with a soft cloth or cotton swab, then thoroughly wiping it down with a non-abrasive antibacterial wipe. 

Headset Electronics 

The preferred way to clean a headset’s electrical parts is by isolating the component and wiping thoroughly with a dry soft cloth or cotton swab.

In addition, the Safety Committee recommends avoiding exposure of devices to UV rays or direct sunlight, reducing sharing of devices when possible, and using personal or disposable covers for high-contact areas, such as brow pads.

Finally, please keep in mind that public health officials are still learning about the coronavirus and any guidance or recommendations may be subject to change. Please consult your jurisdiction’s infectious disease experts for the latest information.

The AREA Safety Committee is dedicated to identifying and prioritizing AR safety risks and recommending steps that enterprises can take to manage and reduce those risks. Visit here to learn more about the Committee, its work, and how to join.

AREA: Where do things currently stand in terms of standards development for AR?

McGarr: There are a few things that are going on in international standards. For example, there are already standards for spatial recognition, amongst other areas. But the reason I’ve been focused on getting new standards going is that some of the barriers to AR becoming mass-market relate to aspects of the technology that are not standardized.

That’s where I see BSI coming in. As the UK national standards body feeding into European and international standards, we span a broad spectrum of standards, from door locks to ethics to cybersecurity and everything in between. For VR and AR to become a mass-market proposition, they will have to interact with the way many industry sectors already successfully work, which is generally based on longstanding areas of standardization. Such standardization is in place to deal with the challenges mature sectors have dealt with in the past. For this reason, the new standards work is focused primarily on ensuring health and safety are taken into account.

Among many industry sectors, such as construction and manufacturing, there are commonly accepted procedures and equipment to ensure health and safety. So even if AR technology delivers increased productivity, if it doesn’t align with existing health and safety procedures or equipment, it’s not going to be accepted or widely used.

There are several aspects to consider; first, ensuring that the technology itself doesn’t introduce new concerns – for example, if a person on a manufacturing floor using VR or AR stops using the technology and feels dizzy in a potentially dangerous environment. Second, the technology has to be interlinked with other standardized personal protective equipment (PPE) – such as protective gloves or hardhats. The technology has to work within those parameters which are generally supported by standardization. Then there are areas of standards that are likely to follow – for example, ensuring that a device is robust enough to work within an environment like a building site or ensuring appropriate cybersecurity and privacy management.

All our work is about taking what is a great technology proven to make organizations more efficient and get it to a place where it is accepted by most mainstream organizations.

AREA: Does BSI begin by identifying which issues are the highest priorities?

McGarr: We started by commissioning independent research to determine where standards broadly were needed. Using that as a starting point, we’ve talked with a lot of people in the market to determine what the main priorities are. So we’ll work on the main ones first and then build further out. In AR, the highest priorities are safety, which includes setup and immersion time, linking up with PPE, and cleanliness, since we have people sharing devices. Cleanliness of devices has been especially important in healthcare, but it has taken on greater importance in all sectors during the pandemic.

AREA: How does that process of industry outreach work?

McGarr: A lot of it is finding and reaching out to the right people. We think about who the main stakeholders are. That can include people from the academic world, government, trade bodies, consumer groups, as well as companies of all sizes. We work to get the right stakeholders around the table and give them the opportunity to represent their views. Part of the reason that we are AREA members is about reaching out to those people, both from the UK and the AREA members from other countries who can feed in locally.

AREA: Tell us about your recent AR and VR standards development workshop.

McGarr: We’ve spent a lot of time determining how best to proceed to get standards that can grow the AR market. International standards committees work in various ways. You can develop a new committee or start a working group within an existing committee. After lots of discussion we have recently had agreed the best approach is in forming a new “Working Group” (WG11) within the ISO/IEC SC24 committee.  This Working Group will be developing standards to build out areas of standards to aid the AR market to grow with standards specific to AR/VR such as Health & Safety, Personal Protective Equipment, hygiene, diversity/equality/inclusion, robustness, content capture/processing/postproduction, and cybersecurity/privacy/online harms. The initial standards proposals to start this work are being voted on and can hopefully start very soon.

AREA: When can we expect to see the fruits of this AR/VR standards development work?

McGarr: We hope to get work started in a few months’ time. The international standards will be there in approximately two to four years. If there is a desire to do things more quickly, there are ways to do them more quickly. It’s all about bringing stakeholders together, bringing in a wide variety of views, building consensus, and establishing best practices.

AREA: If people reading this want to get involved or get more information about your work, what should they do?

McGarr: I’m happy to be the main point of contact. People can reach me at [email protected]. We’ll also be feeding back into the various AREA groups.

Overview

A key component of the AREA’s goal is to help accelerate the growth of a comprehensive AR ecosystem. Our Educate initiative advances this goal by seeking to further engage with academic institutions to provide feedback on how they can help equip the graduates of tomorrow with the AR skills needed to positively contribute to the workforce.

Earlier in 2020, the AREA, together with our academic members, conducted a survey to capture industrial perspectives on both educational needs for future graduates, as well as an assessment of the current state. In this editorial blog, we’d like to share the main findings of the survey results.

Summary

The survey highlighted several key results:

1. There is a high level of agreement, across multiple industries, that AR is either in mainstream use now or will be in the next few years
2. Industry views educating students in AR as important,
3. The business aspects of AR, rather than deep technical knowledge, are deemed to be of higher importance.
4. Recent graduates are typically under-skilled in AR.
5. Industry is willing to engage with academia to help address these challenges.

Survey respondents

The survey attracted 43 respondents, with a good mix between those providing AR solutions and services (58% of respondents) and those using them (42% of respondents). Perhaps more importantly, we captured perspectives from a wide range of industries, as shown in Figure 1, with the highest levels of response from the automotive, industrial equipment and power and energy industrial sectors.

Figure 1 Primary industrial sector of survey respondents

AR adoption now and in the future

When asked “When do you see AR becoming mainstream in your business?”, 20% of our survey respondents stated that AR is already mainstream in their business. Another 60% of respondents believe that AR will become mainstream within 1-2 years, with the remaining 20% suggesting a longer timescale of 3 to 5 years, as shown in Figure 2. Perhaps unsurprisingly, no respondents chose “never.”

Figure 2 Perspectives of when AR will become mainstream

Where should academia focus on educating students?

Our next question, perhaps the most significant of the survey, requested views on where academia should focus on educating the graduates of tomorrow. Respondents were asked to rate each of the following graduate attributes from 1 (least important) to 10 (most important):

• Deep technical familiarity with the underpinnings and principles of AR software or hardware
• A strong theoretical foundation of how AR technologies can benefit industry and society
• Practical experience developing (coding, authoring) AR applications
• Practical experience and understanding how to apply AR technology to “real-world” business challenges
• Familiarity with using AR as a tool in business settings and use cases
• An understanding of the business aspects of AR (costs and cost savings, ROI, safety, security, privacy)
• Awareness of global and industry trends in the adoption and usage of AR
• Experience with the human factors, ergonomics or user interface design of AR solutions

The distilled answers to this question are shown in Figure 3. We order the answers by the number of respondents that suggested a value of between 8 and 10 (inclusive) for each educational need.

The responses clearly show that the “business-oriented” aspects of AR are believed to be more important than the underlying technologies and ergonomics. 75% of all respondents ranked the business aspects with scores of 8 to 10. In particular, 100% of respondents scored the educational need “Practical experience and understanding how to apply AR technology to ‘real-world’ business challenges” with a score of 8 to 10.

Perhaps this reflects an industrial requirement that graduates better understand how to apply AR technologies rather than the ability to build such technologies.

As such, it is clear that the results of our survey highlight a need for academia to equip the graduates of tomorrow with the skills addressing how to apply AR to business challenges and how to quantify and qualify business value, cost and other practical considerations.

Figure 3 Where should academia focus?

The importance of AR in university curricula

When asked “…how important do you believe it is, that academic institutions should include AR in their curricula?”, the answers were as shown in Figure 4.

61.5% of respondents answered, “Very important.” Interestingly, 92% of respondents deemed academic support for AR as either “Very” or “Quite” important.

Figure 4 The importance of AR in university curricula

If these answers reflect a sentiment shared across industries, then there is a clear message to academic institutions to include aspects of AR in various university courses of study.

AR skill levels of recent graduates

We then asked respondents to rate their impressions of the AR skills and experience of recent graduates. The results are shown in Figure 5. Of these results, 42.4% of respondents believed that the skills and experience were either adequate, good or excellent. More worryingly, 45.5% of respondents were of the opinion that the AR skill levels were either poor or non-existent.

The question did not dig into how the skills were acquired (e.g., by way of new-hire training) but nevertheless, the answers clearly represent a set of mixed opinions of how well-equipped recently graduated staff are to embrace and apply AR within the workplace.

Figure 5 Perceptions of AR skills of recent graduates

Industry’s willingness to engage with academic institutions

The last questions of the survey attempted to measure the level of interest expressed by industry to engage with the education process. Figure 6 illustrates an encouragingly high level of willingness to get involved with students in various ways with 70% of respondents willing to propose ideas for course curricula.

Figure 6 Ways in which industry would support student education

There are some clear indications that industry wants to engage further with academia whether it be by sponsoring postgraduate research, suggesting final year projects, or simply suggesting ideas for course curricula. Perhaps more notable from the point of view of the students, there is clearly an appetite for hiring interns with AR skills. 

Finally, when asked if they would be willing to discuss their answers further with AREA staff, 58% of respondents expressed a willingness to do so. We are grateful for their offers to engage further.

Conclusions

Whilst accepting that a response count of 43 is perhaps not statistically significant, the survey results obtained do highlight some key messages for the academic community:

• The data captured by this survey is from a wide range of industries.
• 80% of respondents believe that AR is either in mainstream use now or will be in 1 to 2 years.
• Industry views AR education as important.
• Academic courses should equip students with the knowledge of how to apply AR to business use cases along with other business aspects such as ROI and cost.
• The majority of respondents believe that recent graduates are under-skilled in AR
• Industry is willing to engage with academia to help improve this.

The message is clear: there is a need for AR in academic courses and industry is willing to engage to help make it happen.

Acknowledgements

The AREA team wishes to thank all of those who participated in this survey.

We gratefully acknowledge the assistance of Professor Barbara Chaparro, Embry-Riddle Aeronautical University, in the construction of this survey.

This editorial has been developed as part of the AREA Thought Leaders Network content, in collaboration with selected AREA members.

Corporations in the medical and pharmaceutical industry need to adhere to the highest standards of quality, with accuracy and precision being the keys to success. If organizations experience equipment errors or healthcare workers make mistakes, they not only put human life at risk but also incur significant consequences for payers, including financial and credibility loss. To reduce error rates and increase quality, businesses across the industry are turning to the latest technologies – including Augmented Reality (AR).

AR and VR technology is already being used and significantly improving processes in medical device manufacturing. This editorial discusses two major use cases in which AR solutions simplify workflows to reduce human error:

1. Medical device assembly
2. Production line changeover

Let’s take a closer look at exactly how AR technology can tackle key challenges in both cases, with the help of some first-hand insights from William Harding of industry leader Medtronic, recently interviewed by RE’FLEKT.

Key challenges in medical device manufacturing

Medical device manufacturing typically involves a variety of manual, semi-automatic and automatic processes which makes production particularly vulnerable to error – especially as large manufacturers need to employ the same processes across multiple facilities, often without standardization of production data. In addition, if there is a lack of training among operators it can increase the risk of mistakes made during manual tasks when medical equipment is assembled and configured.

William Harding, Distinguished Fellow at Medtronic, reveals which factors medical device manufacturers need to consider when introducing changes on the production floor:

“If I add a new process to a production line, many questions need to be addressed: How do I get the process to integrate seamlessly (e.g., communication protocols, data aggregation, and data transformation)? How do I accomplish that without using paper-based systems? The goal is to speed up efficiencies and reduce scrap while also reducing human error. When we create a new process in lean manufacturing, we need to establish the most ergonomic way for an operator to perform their tasks within a sterile environment. We also want them to complete these tasks in the most efficient way possible, while delivering a high-quality product. There are many factors to be considered.”

Prior to introducing a new manufacturing process, operators need to be trained on how to perform each step to ensure maximum efficiency and minimum error rates during production. William further explains how Medtronic originally used a cardboard replica of their manufacturing line for training purposes and what challenges came along with it:

“It used to take us two and a half weeks to build a cardboard set-up with five process stations. For one training session, we also needed at least eight to ten people off the production floor, who then weren’t engaged in manufacturing products while they were in training. It would cost us about $30,000 for one training effort with the cardboard set-up. We usually require five sessions in total to get everything right, and by the time we decide that everything is ready, we’re making changes five minutes later.”

Simplified training and operations with AR solutions

With AR technology, medical device manufacturers like Medtronic can not only manage the challenges listed above, but also benefit from significant operational improvements, as the following two use cases reveal.

1. Enhanced AR Training for device assembly and set-up 

Training around medical device set-up and configuration is traditionally based on Operating Procedure (OP) documentation that is not user-friendly. Extensive manuals, including complicated 2D diagrams and text-based instructions, make it challenging to find the right information quickly for device operators. Consequently, onboarding is time consuming and devices may be set up incorrectly and/or not used to their full potential.

Many leaders in the medical sector, including Medtronic, are turning to AR to train employees to set up and assemble their equipment. With results that decrease human errors by 90% and improve training times by 60% (see this white paper for further info) the reasons are obvious. AR training solutions allow device operators to visualize complicated OP documentation in a simple way with the right mixture of videos, text, and images that appear directly in context with the real object. This ensures that device operators always have training content available instantly on their mobile devices, tablets, or smart glasses, thereby experiencing fewer errors during device assembly and set-up.

William from Medtronic shares how AR training guides have replaced the cardboard replica during operator training at Medtronic:

“With content creation platforms like REFLEKT ONE, we can now create AR applications that allow operators to learn a new process by walking through engaging training guides on a tablet instead of using our cardboard model.”

2. Lean production line changeovers with AR-based procedures

When switching the production line from one product to the next, every minute of changeover time comes at the cost of missed revenue as production is down while teams rearrange, set-up, and configure the equipment for the next production cycle. Lean manufacturing strategies can help solve the dilemma to shorten downtime and increase the final output.

AR guidance during changeover procedures results in 40% fewer errors and a 25% faster changeover speed (see this white paper for further info). The interactive guides show operators the ideal state of the task at hand in AR next to the actual state. This way operators can always see what needs to be done as they are working. As augmented instructions guide operators through each step, the risk of error is ultimately reduced for all manual stages of the changeover.

This digitalized process is faster and more reliable as William confirms from his own experience creating AR solutions at Medtronic:

“Recently I created a solution to train operators on a manufacturing process for our Linq II battery bond (an implantable 2 lead EKG data recorder for patients). I made the content available to them online, where they could walk through it themselves and learn how to perform the process using gestures in AR. It’s a very fast and effective way of training because it saves resources and is so close to the real manufacturing environment.”

Outlook: The future of XR technology in the medical sector

These two use cases are great examples of how AR technology is already making a measurable difference in tackling key challenges in training and operations within medical device manufacturing. For the future, William forecasts a growing adoption of AR and ultimately Mixed Reality solutions at Medtronic as well as across the industry:

“Through the use of this technology in the future, I know that Medtronic will be able to more quickly understand the needs of patients, healthcare professionals, and payer’s needs, such that the lifecycles of innovation are reduced in addressing those needs. That same point can be made within the medical device manufacturing industry, specifically as it relates to product and process transfers as well as in the training of the individuals responsible for completing the assembly of those devices. However, it is my belief that AR and eventually MR technology will make the use of VR less important because users will prefer the more relatable MR environments.”