Becoming an expert in equipment assembly and operation requires extensive training and practice. Typically, the operator-in-training begins by studying books and manuals. This learning might be interspersed with instructor-led courses and critical hands-on training.

Once the operator-in-training is deemed “qualified,” he or she is put on the factory floor for on-the-job training. Unfortunately, operators often report feeling ill-equipped to do the job. It is as if much of their prior training was for naught.

The Problem With Current Training

There is a reason for this lack of confidence, and it is rooted in the neuroscience of learning and performance. Constructing a three-dimensional dynamic representation of equipment from two-dimensional training materials requires a huge amount of cognitive effort. First, learners have to hold a mental representation of a series of static images in their short-term (working) memory.

Second, they have to combine them, on the fly, to construct an accurate 3D static representation of the equipment. Finally, they have to infer and impart the dynamic nature of the equipment onto this 3D static representation. Every step in this process is effortful and represents a chance of failure.

Each step also requires an enormous amount of cognitive capacity (in the form of working memory) as well as cognitive energy (in the form of executive attention). Anytime working memory load and executive attentional demands are taxed, we are more likely to make an error and generate an inferior mental representation. Because this cognitive translation process is error-prone, it is impossible to know if the operator-in-training has an accurate representation of the equipment and its operation in his or her head.

Operators-in-training also often have hands-on experience with the equipment. They generally watch an expert complete the steps necessary to assemble and operate a piece of equipment, often with verbal instruction. Next, they are tasked with completing the same steps while receiving corrective feedback from the instructor. This experience is where the real learning takes place.

During this process, the behavioral, experiential and cognitive processing systems of the brain are all active. Simultaneous activation of these learning systems leads to context-rich memory traces that are highly connected and less prone to forgetting. Unfortunately, this type of expert-led learning is time-consuming and costly, and it is not scalable.

Once “certified,” operators are put on the factory floor, where the long-term learning occurs and the ability to deal with adversity develops. This mix of cognitive learning with less frequent, hands-on, expert-led training is suboptimal.

The Solution: Immersive Technology

Immersive technologies like augmented reality (AR) can help. To optimize and increase the efficiency of equipment assembly and operation training, learners need expert-led, hands-on training that simultaneously engages experiential, behavioral and cognitive processing systems. This learning should occur on the factory floor to engage experiential learning systems in the brain that process each sense (occipital lobe for sight, temporal lobes for sound, and parietal lobes for smell and touch).

Within this context, the operators must engage with the equipment to develop the “muscle memories” that link experiential contexts with step-by-step behaviors. Finally, cognitive information provided in this context and in the flow of work via audio or text will engage the prefrontal cortex and, ultimately, reside in long-term memory. This process is how real learning will occur, but companies need a tool that achieves these goals while also being time- and cost-effective and scalable.

Augmented Reality as a Training Tool

Suppose an operator is trained with a hands-free AR tool, such as glasses. She might walk around and view the equipment while receiving verbal or written instructions naming each part and identifying its function. She might touch a specific part and receive more detailed information. The operator might then request training on a specific task, such as disassembling a pump, assembling a control panel or operating the equipment at 70% capacity.

During each of these learning tasks, the operator is in the environment, experiencing the situation (experiential learning). Using visual and auditory assets, the AR tool tells her what to do and when to do it (cognitive learning) so that she learns the steps to follow while performing those steps in real time (behavioral learning) and in the manufacturing environment.

In other words, she learns to operate the equipment by engaging experiential, cognitive and behavioral learning systems in synchrony. Because cognitive and behavioral training co-occurs with visualization of the equipment, as well as the sounds, smells and feel of the equipment, learning is faster and more robust.

The operator-in-training might even be placed in a virtual emergency situation in which she is under a time pressure or where an important tool is missing. This training builds the situational awareness that normally develops on the job, when it can place the facility and its employees at risk. Instead, the operator-in-training can gain experience with these rare but dangerous situations in a safe environment. Her confidence and level of expertise will be higher, and she will achieve them more quickly.

Implementing Augmented Reality

If your organization is interested in using AR tools in your equipment operation and assembly training, a few considerations are in order. First, in most cases, equipment assembly and operation require extensive behavioral interaction. In this case, a hands-free device is likely critical, as it will allow the complete freedom of movement that is central to behavioral skills learning. While hands-free devices are more expensive, hardware costs continue to decline.

In addition, it’s important to consider the human factors of the system. Augmented information can reduce the cognitive load, but it can also overtask the user with unnecessary information. Look for vendors that address these issues and can present data to support the effectiveness of their offering.

Finally, if you are considering a do-it-yourself AR authoring tool, be careful of the risk of cognitive overload. Be sure to conduct good experimental testing and modification to optimize your tool. Your goal is to provide users with what they need, where and when they need it.

With these considerations in mind, you can improve the learning experience, make your employees safer once they’re on the job, and make the worksite a better and more effective place to be.

Industry analysts continue to predict that the Augmented Reality (AR) market will exhibit significant growth and will support a transformation of the ways in which many companies run their businesses. However, as with any new technology, early adopters lead the way and many companies have not yet reaped the business benefits promised by AR.

In this AREA editorial, we take a look at some of the reasons why some companies are not yet fully embracing the potential of Augmented Reality and what the AR ecosystem is doing to overcome these concerns.

In short, what are the current barriers to adoption of Augmented Reality?

We break down this topic into a number of subject domains:

• Business barriers
• Use case applicability and deployment
• IT and security
• Human factors
• Content
• Enterprise systems integration

Business barriers

Key to business adoption and success of any new technology is the ability to focus on the business value of a new technology rather than the technology itself. Augmented Reality vendors need to position their offerings in ways that appeal to senior business buyers and decision makers in order to lower the barrier of uncertainty often faced by executives.

The recent publication by AREA member Atheer of a proposed Maturity Model for AR helps companies paint their own vision for a roadmap to AR adoption by building on the work of those who’ve gone before. It is a useful tool to describe the steps required to deploy AR successfully within their businesses. Geof Wheelwright, Director of Marketing Communications at Atheer, comments “The maturity model articulates how to get started and also how to mature enterprise use of AR to the point where it really does provide a competitive advantage. The AR Maturity Model helps organizations understand each of the four distinct stages of AR Maturity – and the clear steps they need to take to move from one stage to the next.”

AREA member Christine Perey of PEREY Research & Consulting adds:

“New technology adoption requires the active support of business managers. In an ideal scenario, the business line managers as well as the senior management are ready to embrace the new technology. However, most AR introductions suffer from one or more of the following shortfalls in management:

• Lack of leadership, either due to lack of confidence in the team or the technology, or, at the other extreme;
• Company leadership that is advocating for technology introductions without full understanding of timing (and perhaps too early or too quickly);
• Imbalances with (or poorly informed) risk analyses and assessments.”

Despite constructing a seemingly sound value proposition for the adoption of AR technology, projects often flounder for a number of reasons. Christine Perey comments:

“Many AR projects encounter financial barriers: either funding is insufficient and/or the resources are not distributed across all the components of a successful AR adoption campaign. There is also a general shortage of information about the costs and components of return on investment (ROI). Finally, AR product and service providers are searching for new and different business models (Data as a Service, Software as a Service, Hardware as a Service) which can be unfamiliar to customers and need to be tested before proven.”

To help companies create a robust assessment of the ROI offered by AR, the AREA is currently trialling an ROI calculator.

However, challenges remain. Christine Perey adds:

“Despite tools to calculate ROI, there continue to be challenges due to lack of clarity in:

• Final costs of AR hardware, services and software licenses;
• Unique or unusual recurring costs which greatly impact the total cost of ownership; or
• Benefits other than time savings and error reduction that are not easily measured in pilot projects.”

Use case applicability and deployment

A previous AREA editorial discussed the issues related to the fact that AR investigations often stall after development and demonstration of a “cool demo,” highlighting the critical importance of ensuring that investigations into AR are firmly grounded within an appropriate use case and application of the technology that makes sense for the investigating company.

An appropriate tool to use for such evaluations is the “Opportunity versus Readiness” map which, following a set of weighted criteria definitions of potential positive business outcomes and the technical, infrastructure, and other business readiness aspects, enables the initial mapping of potential AR applications to gain a business-centric view of where to focus initial investigations.

An example analysis is shown in Figure 1 with the most feasible and impactful applications appearing in the upper right-hand quadrant.

Figure 1. Example ‘Opportunity versus readiness’ mapping

IT and security

As AR often relies upon the delivery of an enterprise’s Intellectual Property (IP) to new devices and systems that are logically and physically outside the corporate firewall, there clearly must be sufficient measures in place to protect this IP. A recent article explores many of the issues related to cybersecurity with AR, pointing out the uneasy position of AR-related innovation presenting new and ill-understood challenges to corporate IT leadership.

To support understanding of these issues, the AREA’s Security Committee is focused on studying security topics within AR and proposing best practices for adoption by vendors and users of the technology.

Human factors

Owing to the inherent nature of AR, involving both the delivery of visual and aural content whilst in some cases capturing information about the users, often coupled with doing a job differently, a number of concerns have been raised and studied.

Privacy protection is a common concern both legally and at a more personal level. As AR-enabled devices typically use computer vision technology with video feeds from the device camera(s), in addition to capturing sounds, user location, etc., it’s easy to see why there are concerns. However, industrial best practices provide a way to mitigate many of these concerns by ensuring users are aware of what is being captured and how that information is used.

Resistance to change caused by the transformation of an existing role to one that harnesses AR has been shown to be partially offset by involving those impacted in the process of technology adoption. Christine Perey comments:

“Attention to the worker/technician attitudes about AR is extremely important in all stages of the enterprise AR project.

Common steps to reduce resistance include working directly with end users in the design of user interfaces and proving use cases, providing incentives for participation and feedback in AR proof of concept projects and trials, and making sure that there is a highly respected tradesperson or technician on board with the AR introduction plan.”

The wearing or carrying of AR-enabled devices has safety implications. They can cause discomfort or reduce situational awareness, amongst other concerns. A recently published report by Electric Power Research Institute (EPRI) described some key findings and identified areas for further study. Furthermore, the AREA is actively engaged in supporting discussion of these topics, with a number of members participating in the AREA Human Factors Interest Group.

Content

The ready availability of applicable content to underpin scalable deployment of Augmented Reality apps and experiences is an ongoing challenge. Such content spans 3D models, step-by-step sequences for service and manufacturing processes, and more. A typical first foray into Augmented Reality often involves the manual re-modeling of 3D product models to support the AR experience. However, this manual task has inherent cost and time implications, typically raising management concerns about the applicability of AR to support scalable deployment of business processes.

Moreover, within many companies, there are existing 3D CAD files that offer reuse possibilities if suitable transformation, IP protection and optimization could be harnessed within an automation framework.

David Francis, Chief Marketing Officer at Theorem Solutions comments:

 “Many companies will spend absolutely ages re-crafting their 3D CAD into something usable in a games engine, using a product like 3DS MAX. 

However, in doing so, the connection to the initial design is lost.  Moreover, as the correct geometry isn’t even used, the experience may be “representative” rather than actual.”

So, if you are planning on using 3D content in your XR app then it is probably best to find a solution that can make best use of your existing CAD assets.  A solution that can read the geometry as it was designed and will prevent someone having to spend hours redoing something that the designer already did.”

Enterprise systems integration and interoperability

A compelling feature of Industrial AR is the ability to present pertinent content over the physical world. Depending upon the use case, this data, in principle, can be extracted from IoT, ERP, CRM, PLM, or other business systems. This systems integration task is often perceived as difficult and costly, if indeed possible at all.

Additionally, there should be systems in place to deliver content that is relevant, up-to-date, secure and correct. David Francis agrees that when such delivery systems are not available, “Any downstream changes to 3D CAD models will need to be manually re-modeled, as these are maintained outside of the PLM process.” 

However, there is growing support for such enterprise systems integration by vendors in the AR ecosystem but more is needed.  The AREA Interoperability and Standards Program is promoting a workshop in January 2020  to specifically focus upon AR interoperability requirements with the aim of sharing the outcomes with Standards Development Organizations and AR vendors to help accelerate adoption of standards within the AR community.

Conclusion

The following summarizes the points made in this editorial article:

We hope this AREA article has explained some of the current perceived challenges and ways in which they are being addressed. As with many new technologies, the AR market of solutions is rapidly maturing to address these challenges and working hard to help global industries embrace the value offered by AR to transform many business processes.

For further reading, we invite you to read a report, commissioned by the AREA, which examines some of these objections which pertain specifically to the manufacturing sector. 

The AREA actively promotes discussion in a number of the domains discussed in this article by supporting various AREA committees. These comprise AREA members with expertise in the various topic areas who meet on a regular basis to help move the ecosystem forward.

We thank the contributors for their perspectives and welcome your feedback.

The project began in September 2017 and is now concluded.

This recently completed project was funded by the EPRI’s Technology Innovation program (first two years) and in this final year it was supported by members of Program 62-Occupational Health and Safety.  In addition, this project received in-kind support of the augmented reality equipment by EPRI’s Program 161 – Information & Communication Technology – John Simmins of Alfred University and Norm McCullough.

Some of the key findings of this report included:

• A system of hardware, software, and experimental protocol was developed to measure the electromyography (EMG) activity of major neck and shoulder muscles and eye blink rate while two
• groups of utility workers—power plant operators and manhole workers—used the HoloLens and HMT-1 to conduct common tasks.  In addition, we assessed situational awareness.
• The extra weight of the HoloLens and HMT-1 did not, in general, increase the EMG activity of the neck and upper shoulder muscles.
• The HoloLens decreased blink rate 6 to 11 blinks per minute for the manhole workers when they performed two of the tasks: manhole inspection and taping a splice. A decrease in blink rate is an
• indicator of eye strain, which includes dry-eye syndrome.
• The power plant operators rated the HMT-1 easier to use and more comfortable than the HoloLens.  The manhole workers rated the HMT-1 more comfortable than the HoloLens.
• The perceived ratings of situational awareness—that is, the ability to detect objects around the worker and conduct work safely—did not vary significantly between the two AR devices.

An important caveat with these findings is that the tasks were of rather short duration.  As a first of its kind, we have drawn some lessons learned about potential new work.  We recommend that that future studies should be done addressing longer tasks.  Also, AR technologies are advancing and future studies should reflect those advances.  For example, the HoloLens used in this project was 1^st gen.

EPRI previously published a literature review on this topic, which was the most downloaded report in 2018 from EPRI Program 62-Occupational Health and Safety.  This report is still available at www.epri.com and remains in the public domain.

The title of the report is: Program on Technology Innovation: Augmented Reality—Literature Review of Human Factors Issues in the Electric Power Industry. EPRI, Palo Alto, CA: 2018. 3002012532.

The AREA team and on behalf of all our members would like to congratulate all those involved in the project and for making this available in the public domain.

Our two educational tracks are Exoskeletons and Augmented Reality and ErgoX will:

• Provide an update on human factors, safety, and ergonomics research methods for exoskeletons
• Highlight standards and technology on exoskeletons
• Inform and educate stakeholders (designers, users, and researchers) about exoskeleton types, applications, and their potential to reduce injuries and increase human performance (productivity, quality, etc.).

Exoskeletons and Exosuits are wearable technologies designed to augment the human musculoskeletal system to improve physical performance. Their potential value extends across disciplines, and includes improving industrial worker capabilities and enhancing medical rehabilitation.

Augmented reality is an interactive experience of a real-world environment where the objects that reside in the real-world are “augmented” by computer-generated perceptual information, sometimes across multiple sensory modalities, including visual, auditory, haptic, somatosensory, and olfactory.

The ErgoX Symposium will bring together professionals from multiple segments to share their knowledge, experience, and goals:​

1. Designers and producers will showcase and demonstrate their products.
2. Researchers from universities and government agencies will provide information on approaches to the design of human factors research on exoskeletons to appropriately assess fit, usability, safety, ergonomics, productivity and cost-effectiveness.
3. Users and workers with experience using exoskeletons in production environments will share their evaluation methods, expectations, and findings, both positive and negative.

The months of May and June bring a flurry of AR events that
the AREA and its members support. So, with my travel bag in hand and a list of
industry experts with whom I would like to meet and discuss the work of the
AREA, panels to chair and talks about the growth of the enterprise AR
ecosystem, I say goodbye to my family and travel to the US.

VRX Immersive Enterprise – Unleash the Full Potential of
XR in Enterprise – 21/22 May, Boston

I’m always excited to attend enterprise-focused events like
VRX. This was my second year and it continues to grow. The speakers were
excellent, albeit perhaps focusing more on VR than AR. This year I moderated a
panel titled – AR for enterprise – What’s happening now and what does the
future hold?

I was joined by:

• Rich Rabbitz, Lockheed Martin – I’ve been
  lucky to chair a number of panels with Rich, he is an expert and practitioner
  in the AR space.
• Doug House, Porsche Cars of North America
  – Doug and his team have pioneered the use of remote assistance to make service
  at its dealerships hugely more efficient.
• Caroline McManus, PTC – Caroline is a Senior
  Market Strategy Analyst, Augmented Reality and was able to provide real insight
  and experience to the panel.

 We covered the key
use cases being implemented in the industry, the benefits / ROI (both tangible
and intangible) that the companies deploying AR have received, and the
challenges they have had to overcome (including convincing stakeholders to
invest, safety, security and technology issues). A great panel and engaged
audience!

As part of the AREA’s media agreement with VRX, I offered
and ran a mini-master class on Enterprise AR. This intimate, interactive
workshop was a great way to present the work of the AREA. The focus is on
highlighting its members thought leadership, explaining the key use cases
(problems being solved), case studies (examples of companies deploying use
cases) and how to overcome potential barriers to adoption. A good session with
lots of positive feedback!

AWE US – 29– 31 May, Santa Clara

After a few days back in the UK, I once again boarded a
plane to the US – this time, Santa Clara. This was my fourth time at AWE US, an
event that is always well-attended with the expo floor growing year over year.
The number of speaker tracks has also increased.

The AREA was unable to secure a speaking slot this year but
did run a very successful AREA meetup. With an 8 am start on Thursday morning,
coffee and doughnuts fuelled the conversation. Our thanks to AREA member PISON for allowing us to use their
meeting room. Over 50 earlier birds attended and heard from AREA sponsor /
board members. Brian Vogelsang from Qualcomm, Marc Schuetz from PTC, Christine
Perey from PEREY Research and Consulting, Jay Kim from Upskill, and Peter Tortorici
from Medtronic all discussed the experiences, insights, and benefits they have
gained from AREA membership.

We also heard from a few of the AREA Committee chairs (the
AREA runs monthly calls focusing on working together to reduce barriers to AR
adoption including Research, Safety, Security, Human Factors, Marketing and
Requirements). Tony Hodgson from Brainwaive (Security Committee Chair) and Christine
Perey (Research Committee Chair) spoke about the objectives and deliverables
from their respective groups.

Back home for a few days, bags packed and I’m excited to
attend…

LiveWorx 2019 – 11 – 13 June, Boston

It was my first visit to LiveWorx and really enjoyed the
experience! The event is focused on helping companies digitally transform. PTC
CEO (and AREA member) Jim Heppelmann delivered a keynote that covered a wide
range of solutions including the latest PTC AR technology, featuring live demos
of many of PTC solutions. There were also many speaking tracks, including one
dedicated to AR.

I was delighted to present the work of the AREA in a 45-minute
session titled “The State of Enterprise AR.” We drew a sizeable audience, and I
was able to cover many subjects including: the history of AR (using Gartner’s
hype cycle); the key use cases being deployed; and the how the AREA’s research
work – including the AREA
ROI Calculator,  Research
in AR Wearable Security and AR
Human Factors and Safety framework – are helping to overcome barriers to
adoption.

The AREA members wanted to offer an in-depth, insightful and
comprehensive workshop, so I was honoured to be joined by Shelley Peterson
(Lockheed Martin), Mary Claire McLaughlin and Rachel Boykin (Newport News
Shipbuilding) and Chris Ambrose (Strategy Analytics) in running a 90-minute
workshop.

With a focus on interaction, the attendees were able to ask
any questions to the team. We covered a wide range of topics on enterprise AR,
from detailed questions on use cases to ROI and benefits. We also addressed a
wide range of barriers and how these companies had successfully deployed AR. It
was a great session and the feedback was amazing, with everyone who attended
thanking the team for their honest insight and feedback. It was a great way to
finish the event season!

Thank you to the event producers, the companies sponsoring
and the attendees. Our industry continues to grow and with more real-world case
studies and enterprises deploying AR solutions, I look forward to kicking off
the 2019 second-half event season!