Manifest, AR Remote Guidance

The AR technology firm provided direct onboarding during a trial with British Airways, which introduced Taqtile’s products to onsite workers.

Dirck Schou, the CEO of Taqtile, added,

“This unique accelerator program has been a great way to introduce airlines to cutting-edge technologies like Manifest which can help them improve the performance of technicians and engineers immediately”

Additionally, the 10-week programme taught staff how to utilize the Manifest platform. Taqtile’s service operates on spatial devices such as Magic Leap and Microsft’s Hololens 2; the product also works across multiple devices, including tablets and smartphones.

An onsite worker can access guidance resources such as digital manuals, video guides, holograms, and 3D models, all presented as detailed AR visualizations. Manifest displays digital resources in the field of view (FoV) of a worker’s headset, and the wearer can navigate a spatial interface hands-free.

Schou continued, stating,

“Through demonstrations of our AR-enabled work instruction platform over the 10-week program, airline industry leaders have gained a better understanding of the tangible benefits Manifest is capable of delivering”

For frontline airport staff, Taqtile’s solution helps workers learn invaluable company-centric knowledge and enhance their efficiency when performing maintenance tasks.

Taqtile explained how airports could leverage its Manifest solution by dispersed teams providing live guidance from an operations centre to frontline employees.

Manifest supports several file types, including photos, videos, real-time 3D (RT3D) content, computer-aided designs, and PDFs. Taqtile recently teamed up with Microsoft this month to integrate the Azure Remote Rendering platform into Manifest.

The move enables firms to perform large-scale onboarding, training, and operational duties with increased efficiency and engagement. Taqtile and Microsoft achieve this by integrating Azure-powered streaming to enhance RT3D content distribution across Manifest-ready devices.

Key updates in this release:

• Advanced Model Target Improvements: Training times for Advanced Model Targets have been optimized and now depend on the number and size of views. Recognition performance for advanced, close-up views has also been improved.
• Area Target Improvements:
  • The target’s occlusion mesh is now exposed in the C API which allows native apps to render occluded virtual content in combination with Area Targets as you move through the space.
  • Textured authoring models are now created by the Area Target Creator app and the Area Target Capture API providing an improved authoring experience in Unity. These scans can be loaded into the Area Target Generator for clean-up and post-processing.
  • Area Target tracking data is now compressed and takes up to 60% less space.

• Unity Area Target Clipping: Area Target previews in the Unity Editor can be clipped based on height, for faster previewing and better visibility of the scanned space during app development.
• Engine Developer Portal (EDP) Self-Service OAuth UI: OAuth Engine credentials can now be managed from the EDP, eliminating the need for the command line.
• Notices
  • High Sensor-Rate Permission: Due to new Android permission requirements, developers should add the “high sensor rate” permission to all native Vuforia Engine apps running on Android 12+ for all releases, otherwise VISLAM may not work. Read more about VISLAM tracking here.
  • Permission Handling: The Vuforia Engine behavior of triggering OS-specific user permission requests at runtime is deprecated in 10.8 and will be removed in an upcoming release.  All native apps should be updated to manage permissions themselves. The 10.8 sample apps share best practices for this.

Combining Magic Leap’s advanced spatial computing platform with NavVis’s mobile mapping systems and spatial data platform, the two companies aim to enhance the use of AR applications across key industries, including automotive, manufacturing, retail and the public sector.

As part of this strategic partnership, NavVis will bring its NavVis VLX mobile mapping system and NavVis IVION Enterprise spatial data platform to Magic Leap’s new and existing enterprise customers with an initial focus on manufacturing. Magic Leap customers will be able to leverage NavVis’s expansive visualization capabilities to generate photorealistic, accurate digital twins of their facilities at unprecedented speed and scale.

The market opportunity for digital twins and other forms of advanced visualization is significant – with demonstrated potential to transform the world of work as we know it. While attention around the potential of the metaverse has put a greater focus on all types of mixed reality technology, AR represents an immediate opportunity for businesses to enhance productivity and improve operational efficiency. Magic Leap’s open, interoperable platform will also enable the metaverse to scale for enterprise applications.

While the Magic Leap 2 platform offers cutting-edge scanning and localization capabilities in real-time on the device itself, NavVis’s technology will allow Magic Leap customers to pre-map and deploy digital twins in large, complex settings that can cover up to millions of square feet – including but not limited to warehouses, retail stores, offices and factories – for a variety of use cases, such as remote training, assistance and collaboration. Such applications will enable companies to reduce operational costs, enhance overall efficiency and democratize the manufacturing workforce of tomorrow.

“We are seeing significant demand for digital twin solutions from our enterprise customer base and are thrilled to partner with NavVis to make our shared vision for large-scale AR applications a reality,” said Peggy Johnson, CEO of Magic Leap. “Coupled with our Magic Leap 2 platform, NavVis’s advanced visualization capabilities will enable high-quality, large-scale and novel AR experiences that business users demand.”

The NavVis partnership is an essential component of Magic Leap’s strategy to cultivate an ecosystem of best-in-class technology partners that will deliver on the promise of enterprise AR, leveraging Magic Leap 2’s powerful, open platform. With a global customer base of more than 400 companies, including the likes of BMW, Volkswagen, Siemens and Audi, NavVis has a proven track record of delivering immediate and long-term value to enterprises looking to modernize their operations.

“Enterprise AR solutions for larger-scale activations will open the door for greater innovation in the workplace,” said Dr. Felix Reinshagen, CEO and co-founder of NavVis. “Our own experience shows that 3D mapping and digital twins are a fundamental foundation for large-scale persistent AR applications. We’re experiencing strong demand across many verticals with industrial manufacturing as a clear front runner. Magic Leap is a world leader in delivering impactful, innovative experiences in these verticals, and we are excited to collaborate with the company to advance this mission and further enable the future of work.”

About Magic Leap

Magic Leap, Inc.’s technology is designed to amplify human potential by delivering the most immersive Augmented Reality (AR) platform, so people can intuitively see, hear, and touch digital content in the physical world. Through the use of our advanced, enterprise-grade AR technologies, products, platforms, and services, we deliver innovative businesses a powerful tool for transformation.

Magic Leap, Inc. was founded in 2010, is proudly headquartered in South Florida, with eight additional offices across the globe.

About NavVis

Bridging the gap between the physical and digital world, NavVis enables service providers and enterprises to capture and share the built environment as photorealistic digital twins. Their SLAM-based mobile mapping systems generate high-quality data with survey-grade accuracy at speed and scale. And with their digital factory solutions, users are equipped to make better operational decisions, boost productivity, streamline business processes, and improve profitability. Based in Munich, Germany, with offices in the United States and China, NavVis has customers worldwide in the surveying, AEC, and manufacturing industries.

In the world of pharma manufacturing, precision is key. To execute flawlessly, pharmaceutical scientists and operators need the proper training and tools to accomplish the task. User-friendly augmented reality (AR) and mixed reality (XR) technology that can provide workflow guidance to operators is invaluable, helping name brand companies get drugs, vaccines, and advanced therapies to patients faster.

AR has been a cost-effective way to improve training, knowledge transfers, and process execution in the lab during drug discovery and in the manufacturing suite during product commercialization. Apprentice’s AR Research Department is now seeing greater demand within the pharma industry for XR software capabilities that allow life science teams to use 3D holograms to accomplish tasks.

For example, operators are able to map out an entire biomanufacturing suite in 3D using XR technology. This allows them to consume instructional data while they work with both hands, or better understand equipment layouts. They can see and touch virtual objects within their environment, providing better context and a much more in-depth experience than AR provides.

Users can even suspend metadata in a 3D space, such as the entrance to a room, so that they can interact with their environment in a much more complete way, with equipment, objects and instruments tethered to space. Notifications regarding gowning requirements or biohazard warnings for example will automatically pop up as the operator walks in, enriching the environment with information that’s useful to them.

“It’s all about enhancing the user experience,” Linas Ozeratis, Mixed Reality Engineer at Apprentice.io. “At apprentice, our AR/XR Research Team has designed pharma-specific mixed-reality software for the HoloLens device that will offer our customers an easier, more immersive experience in the lab and suite.”

Apprentice’s XR/AR Research Team is currently experimenting with new menu design components for the HoloLens device that will reshape the future of XR user experiences, making it easier for them to interact with menus using just their fingers.

Apprentice’s “finger menu” feature allows users to trigger an action or step by ‘snapping’ together the thumb and individual fingers of the same hand. Each finger contains a different action button that can be triggered at any time during an operator’s workflow.

“Through our research, we’ve determined that the fingers are an ideal location for attaching AR buttons, because it allows users to trigger next steps without their arm or hand blocking the data they need,” Ozeratis added.  It’s quite literally technology at your fingertips.”

Why does the pharma industry want technology like this? Aside from the demand, there are situations where tools like voice commands are simply not feasible. The AR Research Team also learned that interactive finger menus feel more natural to users and can be mastered quickly. Life science teams are able to enhance training capabilities, improve execution reliability and expand the types of supporting devices they can apply within their various environments.

“Introducing these exciting and highly anticipated XR capabilities is just one stop on our roadmap,” Ozeratis adds. “There are bigger and bolder things ahead that we look forward to sharing as the pharma industry continues to demand more modern, intelligent technologies that improve efficiency and speed.”

AREA: Tell us about your background in AR at Toyota.

Kleiner: I started almost three years ago. I was working at our headquarters, so my interest in AR was more around marketing and sales than engineering and design. I was convinced that AR was going to be the next big thing in marketing and I wanted Toyota to get ahead of the market.

We made a couple of prototypes that never made it past the prototype stage but got some good reviews. Then COVID hit and everyone was saying, “We’re in a touchless society now; how do we sell?” Well, this is how you sell. As a result, what was a part-time hobby job for me is now full-time. What I’m trying to solve now is how XR fits in at every stage of the customer journey. We’re trying to educate brand-loyal customers that we’re cutting edge and that there are some Toyota vehicles coming out that you can experience at home without visiting a dealership. We’re then trying to track that through to a sale.

A lot of this is about customer education. They don’t know, for example, how Toyota Safety Sense works. They don’t know all the features we put into the vehicle. So our goal is to educate them and “wow” them at the same time – which we hope leads to a sale. Once the sale is made, how do we do the “care and feeding” afterwards? We already have a prototype of an AR owner’s manual, which went over really well – and also reduces costs because there’s no printing required. On our first project, the Lexus AR Kit, we found we could save $11 million a year on shipping and handling of owner’s manuals.

Once we’ve made the sale and delivered the owner’s manual, the next step is selling accessories – and that’s the other product I’m working on right now. We want to be able to capture the buyer’s car – not a cartoon, but a real image – and fly in some accessories to show them how they can customize their vehicle to look better and perform better.

That’s where we’re going. We’re trying to follow the full customer lifecycle.

Our first proof of concept, as I mentioned, was the Lexus AR Kit. It started out as a training platform for dealership personnel but then we modified it to train people on how to use their vehicles. The second pilot we did was a Virtual Showroom. Users can plop the car down at scale in their driveways, then approach it and look inside the vehicle without having to go to the dealership. We completed that in mid-September. The third pilot we’re working on is for accessory sales. The target product is the Tacoma, our smaller pickup truck, which is the most customized of our vehicles. We believe we can increase Tacoma accessory sales by $5 million. If that’s successful, we’ll roll it out to our other highly-customized vehicles.

I’m also tangentially involved in some training work at our manufacturing plants. Since people can’t come to a classroom anymore because of the pandemic, we’re training them remotely using HoloLens through our partnership with Microsoft. Some of the early results are just phenomenal. We found that we could take training from 13 days down to two days. Retention is better and quality is up. It’s been great for both new hires and for reskilling existing workers to take on new tasks.

AREA: What AR software and tools are you using?

Kleiner: On the training side, it’s all Microsoft HoloLens. For marketing and sales, we don’t have an AR studio, so we’re working with an outside vendor called Groove Jones and it’s all custom.

AREA: Tell us about your experience trying to champion this technology in a corporate setting.

Kleiner: I spent a lot of time trying to educate folks, telling them this is not a toy, this is a true tool that belongs in the enterprise and should be part of the customer experience. And many people basically said, “That’s nice, but we’re going to keep doing things as we have.” Once COVID hit, those same people realized we couldn’t keep doing things the way we always had. I’m in demand now!

AREA: So what’s next for you?

Kleiner: Right now, we’re focused on bringing these current proofs of concept to fruition. If those work out well, we’ll expand to other projects. The AR approach is ideal for demonstrating things that aren’t clear in the owner’s manual – for example, how to install an infant seat in the car. That’ll probably be in the next iteration of the AR owner’s manual project.

In accordance with public health directives to reduce the spread of COVID-19, the AREA has made the difficult decision to postpone its 4^th annual AREA Workshop, originally scheduled to take place in Reston at the end of this month. We regret having to make this decision and apologize to all attendees for any inconvenience.

The AREA workshop is one of the key highlights on the AREA calendar, creating an opportunity for enterprise AR experts and companies looking to understand and invest in AR, to discuss, in a collaborative workshop environment, the benefits and challenges of implementing enterprise AR.

The AREA is looking into rescheduling the workshop in an AR/VR environment in the coming months. Please subscribe to the AREA social media channels for further updates.

Enterprise Augmented and Mixed Reality (AR/MR) have made great leaps in assisting frontline workers across industries by delivering information via wearable devices. The challenge for many organizations is how to get beyond one-time gains in productivity and create sustainable value with AR/MR.

Many industry observers believe the key is to combine AR/MR technology with Artificial Intelligence (AI). AI-driven visual guidance has the potential to continuously improve work outputs and lead to the exponential productivity gains industrial companies are looking for.

On Wednesday, April 1^st at noon ET (9:00 PT/17:00 UK/18:00 CET) the AREA will host a webinar on the subject led by AREA member and Augmentir CEO and co-founder Russ Fadel. Mr. Fadel will explain how AI-enriched AR/MR experiences enable organizations to:

• Empower frontline workers with personalized and dynamic work instructions to perform at their peak productivity
• Close the skills gap by capturing tribal knowledge from experts and making that knowledge a scalable corporate asset
• Deliver AI-based insights and recommendations based on human worker data to help drive continuous improvement across the entire organization – from operations to training to quality.

This webinar is free, but you must register to attend. Don’t miss this opportunity to discover how organizations are leveraging AI to supercharge their AR/MR initiatives and create sustainable value!