Augmented Reality, and how it converges with IoT, AI, Cloud and Edge Computing technologies, was among the loudest and brightest themes of Mobile World Congress (MWC) Barcelona 2019.

The post Augmented Reality a hot topic at MWC Barcelona 2019 appeared first on AREA.

Software

Over the past decade, early adopters of enterprise Augmented Reality like Airbus (formerly EADS), Daimler, Volkswagen, Boeing and others invested in internal software development tools and built their own platforms in order to integrate with existing data and business systems, and to manage as many factors as possible without relying on outside vendors.

Today, many who are starting or managing AR projects are still in research and development phases and seek to control variables as much as possible. They may still choose to build a new solution entirely within their IT organizations, but more often are able to license core enablers from commercial providers (at a cost or without fee), or to use open source projects.

As commercial solutions improve and offers expand, companies frequently find themselves needing to carefully re-evaluate the cost of building (or continuing to maintain) a solution internally. In some cases, purchasing off-the-shelf software solutions for experience development is less costly and there are other benefits.

Commercially ready SDKs for enterprise AR have several advantages. They usually benefit from a better user interface for use by developers. There is almost always documentation or webinars for getting new users up to speed and producing experiences quickly. Commercial solutions may also have more investment in core enabling technologies for object recognition, tracking and rendering. Getting these working with a variety of hardware platforms and maintaining those is costly.

The reason that many enterprises are still in the position of building or developing internally has to do with the risks associated with a commercial SDK provider changing strategies, a lack of resources for maintaining R&D, a low commitment to rapid change needed by enterprise customers, a lack of support for the enterprise IT group’s choice of hardware and maintenance costs of software and extensions. Many licenses have to be renewed annually or base a fee on the number of experiences; if the organization uses AR in many projects, the fees may be greater than developing and maintaining an internally engineered SDK.

Hardware

Any AR project requires sensors for detecting user context and intentions, a display system (personal or shared/stationary and attached to a tool or area of the shop floor) for providing experiences and processing for highly demanding computational tasks.

Since positioning the AR experience is so critical to success, many companies develop custom positioning and user detection systems involving infrared, highly calibrated systems that frequently re-use parts or build upon previous Virtual Reality project components.

Depth sensing is highly appropriate in enterprise AR where many targets are three dimensional and the CAD models are as well. Rendering 3D models in real time is highly processor intensive and benefits from storing the models in memory, another costly hardware component if a project exceeds one or two prototypes and begins to be deployed in dozens of positions.

Most enterprise AR projects begin with tablet computers. Tablets are suitable for prototyping but there are several types of obstacles. One has to do with security and safety using systems that are not designed for industrial environments or data access. For example, many IT departments cannot use devices that require connecting to the Apple App Store.

Separately there is ergonomics. To perform many AR-assisted tasks requires both the user’s hands or for the user to move around in a space without cables or obstructions. Here the options expand and many solutions are worthy of exploration before selecting one that will be deployed as an internal standard.

Some groups choose to develop custom hardware (contracted or internally developed) to meet their precise needs rather than purchase commercial products.

Content

In order to design the AR experience, the original content and models may need to be ported or adapted prior to use. In addition to raw data, there is also the user interface that has to be designed and adapted to a project.

Once in use, content must also evolve as the processes change. This can require starting over entirely. There also needs to be processes for data management and to store the data that has been captured by the AR system during use.

In general, project costs for content development are at least as great as for the other two essential components.

Integration

Once a pilot or prototype is proven, there will be a need to integrate AR-assisted components into the larger enterprise workflow. This requires planning and heavy testing to ensure that other components and procedures are stable and are not sacrificed for the purpose of real time data visualization in AR view.

Integration is a cascading problem, often leading to indirect costs such as upgrades to a database, greater data storage capacity, higher throughput networks and other costs to lower the delay and increase system performance.

Policies and Processes

Augmented Reality is likely to require the development of new policies and procedures and the refinement of existing ones. This involves legal staff, risk management, safety and security staff and other groups with responsibilities outside those of the information technology or business line management groups directly involved in introducing AR-assisted systems.

Policy and process changes may require coordination with regulatory bodies and other companies and organizations. These changes can escalate in terms of both cost and the time, and introduce delays in the calculation of return on original investment.

Training

As the rate of technology introduction and process changes increases, so do employee training requirements. Training is not only for those introducing and managing software, hardware and content, but also for those working in evolving workflows. New steps are introduced and old steps are eliminated.

New or changed training means less time to perform mission critical tasks and more administration. As with costs associated with changing policies and procedures, changes in training can escalate quickly.

Although it may not be possible to measure them precisely, training costs associated with introducing AR in an enterprise could add up in unforeseen areas and be delayed due to courseware development and review costs.

Introducing AR is a Long-term Investment

To truly integrate Augmented Reality in an enterprise will be a long-term process and significant investment, probably comparable to the computerization and automation of many enterprise processes over the past 20 years. These costs must be considered in light of the total impacts on the business.

Management will need the support of AR experts and IT groups to take in all the related costs as well as to estimate the risks of not introducing AR in an acceptable time frame on a large scale.

How does your organization calculate the total cost of ownership for introducing new technology? Is AR introduction stalled by unforeseen cost centers?

The post Total Cost of Ownership of Enterprise Augmented Reality appeared first on AREA.

As I discussed in a LinkedIn Pulse post about AR apps, we cannot expect users to run a different app for each real world target they want to use with AR or one monolithic AR application for everything in the physical world. It is unscalable (i.e., far too time-consuming and costly). It’s unclear precisely when, but I’m confident that we will, one day, rely on systems that make content ready for AR presentation as a natural result of digital design processes.

The procedures or tools for automatically converting documentation or any digital content into AR experiences for enterprise use cases are not available. Nor will they emerge in the next 12 to 18 months. To begin the journey, companies must develop a path that leads from current procedures that are completely separate from AR presentation to the ideal processes for continuous AR delivery.

Leaders need to collaborate with stakeholders to focus on areas where AR can make a difference quickly.

Boiling the Ocean

There are hundreds of AR use cases in every business. All AR project managers should maintain a catalog of possible use cases. Developing a catalog of use cases begins with identification of challenges that are facing a business. As simple as this sounds, revealing challenges increases exposure and reduces confidence in existing people and systems. Most of the data for this process is buried or burned before it escapes. Without data to support the size and type of challenges in a business unit, the AR advocate is shooting in the dark. The risk of not focusing on the best use case and challenges is too high.

There need to be initiatives to help AR project managers and engineers focus on the problems most likely to be addressed with AR. Organizational change would be a likely group to drive such initiatives once these managers are, themselves, trained to identify the challenges best suited for AR.

In 2017, I expect that some systems integration and IT consulting companies will begin to offer programs that take a methodical approach through the AR use case development process, as part of their services to clients.

Prioritization is Key

How do stakeholders in a company agree on the highest priority content to become AR experiences for their top use cases? It depends. On the one hand there must be consistent AR technology maturity monitoring and, in parallel, the use case requirements need to be carefully defined.

To choose the best use case, priorities need to be defined. If users perceive a strong need for AR, that should weigh heavily. If content for use in the AR experience is already available, then the costs and time required to get started will be lower.

A simple method of evaluating the requirements appears below. Each company needs to define their own priorities based on internal drivers and constraints.

A simple process for prioritizing AR use cases (Source: PEREY Research & Consulting).

Markers Won’t Stick

One of the current trends in enterprise AR is to use markers as the target for AR experiences. Using computer vision with markers indicates to the user where they need to point their device/focus their attention, consumes less power and can be more robust than using 3D tracking technologies in real-world conditions.

However, for many enterprise objects that are subject to sun, wind and water, markers are not a strategy that will work outside the laboratory. Those companies that plan to use AR with real-world targets that can’t have markers attached need to begin developing a new content type: trackables using natural features.

In 2017 more enterprise AR project managers will be asking for SDKs and tools to recognize and track the physical world without markers. For most, the technologies they will test will not meet their requirements. If well managed, the results of testing in 2017 will improve the SDKs as suggested in our post about AR software.

The AR Ecosystem and Technology are Immature

While the title of this post suggests that enterprise content is not in formats and associated with metadata to make AR experiences commonplace, the reverse statement is also true: not all the required AR components are ready for enterprise introduction.

Projects I’ve been involved with in 2016 have shown that while there are a few very solid technologies (e.g., tracking with markers on print), most components of AR solutions with which we are working are still very immature. The hardware for hands-free AR presentation is one area that’s changing very rapidly. The software for enterprise AR experience authoring is another. As more investments are made, improvements in the technology components will come, but let’s be clear: 2017 will not be the year when enterprise AR goes mainstream.

For those who have seen the results of one or two good proofs of concept, there will be many people who will need your help to be educated about AR. One of the important steps in that education process is to participate in the activities of the AREA and to share with others in your company or industry how AR could improve workplace performance.

When your team is ready to introduce AR, call in your change management group. You will need all the support you can get to bring the parts of this puzzle together in a successful AR introduction project!

Do you have some predictions about what 2017 will bring enterprise AR? Please share those with us in the comments to this post. 

The post The AR Market in 2017, Part 4: Enterprise Content is Not Ready for AR appeared first on AREA.

There are some who advocate for integrating AR directly and deeply into enterprise content management and delivery systems in order to leverage the IT systems already in place. Integration of AR features into existing IT reduces the need for a separate technology silo for AR. I fully support this school of software architecture. But, we are far from having the tools for enterprise integration today. Before this will be possible, IT groups must learn to manage software with which they are currently unfamiliar.

An AR Software Framework

Generating and presenting AR to users requires combining hardware, software and content. Software for AR serves three purposes:

1. To extract the features, recognize, track and “store” (manage and retrieve the data for) the unique attributes of people, places and things in the real world;
2. To “author” interactions between the human, the digital world and real world targets found in the user’s proximity, and publish the runtime executable code that presents AR experiences; and
3. To present the experience to, and manage the interactions with, the user while recognizing and tracking the real world.

We will see changes in all three of these segments of AR software in 2017.

Wait, applications are software, aren’t they? Why aren’t they on the list? Before reading further about the AR software trends I’m seeing, I recommend you read a post on LinkedIn Pulse in which I explain why the list above does not include thousands of AR applications.

Is it an AR SDK?

Unfortunately, there is very little consistency in how AR professionals refer to the three types of software in the framework above, so some definitions are in order. A lot of professionals just refer to everything having to do with AR as SDKs (Software Development Kits).

In my framework AR SDKs are tools with which developers create or improve required or optional components of AR experiences. They are used in all three of the purposes above. If the required and optional components of AR experiences are not familiar to you, I recommend reviewing the post mentioned above for a glimpse of (or watching this webinar for a full introduction to) the Mixed and Augmented Reality Reference Model.

Any software that extracts features of the physical world in a manner that captures the unique attributes of the target object or that recognizes and tracks those unique features in real time is an AR SDK. Examples include PTC Vuforia SDK, ARToolkit (Open Source SDK), Catchoom CraftAR SDK, Inglobe ARmedia, Wikitude SDK and SightPath’s EasyAR SDK. Some AR SDKs do significantly more, but that’s not the topic of this post.

Regardless of what it’s called, the technology to recognize and track real world targets is fundamental to Augmented Reality. We must have some breakthroughs in this area if we are to deliver the benefits AR has the potential to offer enterprises.

There are promising developments in the field and I am hopeful that these will be more evident in 2017. Each year the AR research community meets at the IEEE International Symposium on Mixed and Augmented Reality (ISMAR) and there are always exciting papers focused on tracking. At ISMAR 2016, scientists at Zhejiang University presented their Robust Keyframe-based Monocular SLAM. It appears much more tolerant to fast motion and strong rotation which we can expect to see more frequently when people who are untrained in the challenges of visual tracking use wearable AR displays such as smart glasses.

In another ISMAR paper, a group at the German Research Center for Artificial Intelligence (DFKI) published that they have used advanced sensor fusion employing a deep learning method to improve visual-inertial pose tracking. While using acceleration and angular velocity measurements from inertial sensors to improve the visual tracking has been promising results for years, we have yet to see these benefits materialize in commercial SDKs.

Like any software, the choice of AR SDK should be based on project requirements but in practical terms, the factors most important for developers today tend (or appear) to be a combination of time to market and support for Unity. I hope that with support for technology transfer with projects like those presented at ISMAR 2016, improved sensor fusion can be implemented in commercial solutions (in the OS or at the hardware level) in 2017.

Unity Dominates Today

A growing number of developers are learning to author AR experiences. Many developers find the Unity 3D game development environment highly flexible and the rich ecosystem of developers valuable. But, there are other options worthy of careful consideration. In early 2016 I identified over 25 publishers of software for enterprise AR authoring, publishing and integration. For an overview of the options, I invite you to read the AREA blog post “When a Developer Needs to Author AR Experiences.”

Products in the AR authoring group are going to slowly mature and improve. With a few mergers and acquisitions (and some complete failures), the number of choices will decline and I believe that by the end of 2017, fewer than 10 will have virtually all the market share.

By 2020 there will be a few open source solutions for general-purpose AR authoring, similar to what is available now for authoring Web content. In parallel with the general purpose options, there will emerge excellent AR authoring platforms optimized for specific industries and use cases.

Keeping Options for Presenting AR Experiences Open

Today the authoring environment defines the syntax for the presentation so there’s really little alternative for the user than to install and run the AR execution engine that is published by the authoring environment provider.

I hope that we will see a return of the browser model (or the emergence of new Web apps) so that it will be possible to separate the content for experiences from the AR presentation software. To achieve this separation and lower the overhead for developers to maintain dozens of native AR apps, there needs to be consensus on formats, metadata and workflows.

Although not in 2017, I believe some standards (it’s unclear which) will emerge to separate all presentation software from the authoring and content preparation activities. 

Which software are you using in your AR projects and what are the trends you see emerging?

Next: Navigating the way to continuous AR delivery

The post The AR Market in 2017, Part 3: Augmented Reality Software is Here to Stay appeared first on AREA.

There’s a great deal of attention being paid to the new, wearable displays for Augmented Reality. Hardware permits us to merge the digital and physical worlds in unprecedented ways. Wearable hardware delivers AR experiences while the user is also able to use one or both hands to perform tasks. The tendency to pay attention to physical objects is not unique to AR industry watchers. It is the result of natural selection: genes that gave early humans the ability to detect and respond quickly to fast moving or bright and unusual objects helped our ancestors survive while others lacking those genes did not.

Although this post focuses on the hardware for Augmented Reality, I don’t recommend focusing exclusively on advancements in AR hardware when planning for success in 2017. The hardware is only valuable when combined with the software, content and services for AR in specific use cases.

Now, considering primarily AR hardware, there are important trends that we can’t ignore. This post only serves to highlight those that, in my opinion, are the most important at an industry-wide level and will noticeably change in 2017.

Chips accelerate changes

Modern Augmented Reality hardware benefits hugely from the continued reduction in size and cost in hardware components for mass market mobile computing platforms. We need to thank all those using smart phones and watches for this trend.

As the semiconductor manufacturers gain experience and hard-code more dedicated vision-related computation into their silicon-based mix, performance of complete AR display devices is improving. Combined with the technology Intel recently acquired from Movidius (which will produce significant improvements in wearable display performance beyond 2017), Intel RealSense is an example of a chip-driven technology to monitor. Other offerings will likely follow from NVIDIA and Apple in 2017.

When available for production, the improvements in semiconductors for wearable AR devices will be measurable in terms of lower latency to recognize a user’s environment or a target object, less frequent loss of tracking, higher stability in the digital content that’s rendered, lower heat and longer battery life. All these are gradual improvements, difficult to quantify but noticeable to AR experts.

As a result of optimization of key computationally-intensive tasks (e.g., 3D capture, feature extraction, graphics rendering) in lower cost hardware, the next 12 to 18 months will bring new models of AR display devices. Not just a few models or many models in small batches.

These next-generation wearable display models with dedicated silicon will deliver at least a basic level of AR experience (delivery of text and simple recognition) for an entire work shift. Customers will begin to place orders for dozens and even, in a few cases, hundreds of units.

Optics become sharper

In addition to semiconductors, other components will be changing rapidly within the integrated wearable AR display. The next most important developments will be in the display optics. Signs of this key trend were already evident in 2016 – for example, when Epson announced the OLED optics designed for the Moverio BT-300.

It’s no secret that over the next few years, optics will shrink in size, drop in weight and demand less power. In 2017, the size and weight of fully functional systems based on improved optics for AR will decline. Expect smart glasses to weigh less than 80gms. Shrinking the optics will make longer, continuous and comfortable use more likely.

Developers raised issues about color quality and fidelity when testing devices introduced in 2015 and 2016. Color distortion (such as an oil spill rainbow effect) varies depending on the type of optics and the real world at which the user’s looking (the oil spill pattern is particularly noticeable on large white surfaces). The 2017 models will offer “true” black and higher fidelity colors in a wider range of settings. Again, the experts will feel these improvements first and “translate” them to their customers.

Another key area of improvement will be the Field of View. Some manufacturers will announce optics with 50° diagonal (a few might even reach 80° diagonal) in 2017. When combined with advanced software and content, these changes in optics will be particularly important for making AR experiences appear more realistic.

Combined with new polychromatic materials in lenses, lower weight and stronger material in the supports, optics will be more tolerant of changes in environmental conditions, such as high illumination, and will fit in more ruggedized packages.

More options to choose from

Speaking of packaging, in 2016 there are three form factors for AR displays:

• monocular “assisted reality” hardware that clips onto other supports (frames) or can be worn over a user’s ear,
• smart glasses that sit on the user’s nose bridge and ears, and
• head-worn displays that use straps and pads and a combination of ears, noses and the user’s skull for support.

The first form factor does not offer an immersive experience and isn’t appropriate for all use cases, but assisted reality systems have other significant advantages (e.g., lower cost, longer battery life, lighter weight, easy to store) so they will remain popular in 2017 and beyond.

At the opposite end of the spectrum, the highly immersive experiences offered by head-worn devices will also be highly appealing for different reasons (e.g., depth sensing, accuracy of registrations, gesture-based interfaces).

We need to remember that the use cases for enterprise AR are very diverse and so can be the displays available to users. The new wearable AR display device manufacturers entering the fray in 2017 will stay with the same three general form factors but offer more models.

In addition to diversity within these three form factors there will be extensions and accessories for existing products – for example, charging cradles, corrective lenses, high fidelity audio and materials specifically designed to tolerate adverse conditions in the workplace environment.

The results of this trend are likely to include:

• those selling wearable displays will be challenged to clearly explain new features to their potential customers and translate these features into user benefits,
• those integrating AR displays will be more selective about the models they support, becoming partners with only a few systems providers (usually leaning towards the bigger companies with brand recognition)
• buyers will need to spend more time explaining their requirements and aligning their needs with the solutions available in their budget range.

Wearable display product strategists will realize that with so many use cases, a single user could need to have multiple display models at their disposal. One possible consequence of this trend could be reduced emphasis on display systems that are dedicated to one user. We could see emergence of new ways for multiple users in one company or group to reserve and share display systems in order to perform specific tasks on schedule.

Rapid personalization, calibration and security will offer new opportunities to differentiate wearable AR display offerings in 2017.

Enterprise first

All of these different form factors and options are going to be challenging to sort out. Outside enterprise settings, consumers will not be exposed to the hardware diversity in 2017. They simply will not invest the time or the money.

Instead, companies offering new hardware, even the brands that have traditionally marketed to mass market audiences, will target their efforts toward enterprise and industrial users. Enterprises will increase their AR hardware budgets and develop controlled environments in which to compare AR displays before they can really make informed decisions at corporate levels. Third party services that perform rigorous product feature evaluations will be a new business opportunity.

While this post highlights the trends I feel are the most important when planning for success with AR hardware in 2017, there are certainly other trends on which companies could compete.

To learn more about other options and trends in wearable AR displays in 2016, download the EPRI Technology Innovation report about Smart Glasses for AR in which I offer more details.

What are the trends you think are most important in AR hardware and why do you think they will have a significant impact in 2017?

Next: AR software matures and moves toward standardization

The post The AR Market in 2017, Part 2: Shiny Objects Attract Attention appeared first on AREA.

That’s the first step. But you know enough not to believe everything you read or see in a YouTube video.

The next step, if you haven’t done so already, is to train yourself to separate the biggest hype from the facts. This is not easy, but you should be able to hit this milestone by attending industry events where AR is being demonstrated and you can put your hands on the products in action, even under highly controlled conditions. Visiting one or more of the AREA members in their offices or inviting them to visit your facility will be even more valuable.

You’ll see some mock ups and, if you ask tough questions, you will also see some of the weaknesses and begin to glimpse the complexity of the problems facing adoption of these technologies. Keep a log of these experiences you have with Augmented Reality and the impressions they leave on you.

If you really want to understand the strengths and weaknesses “up close” and have budget, you can develop a project or participate in a group project that focuses on a well-defined use case.

Share what you learn

Once you’ve seen and captured notes about more than 10 live demonstrations in your journal and have personally touched AR, you can begin to “translate” for others what you’re seeing and doing.

But, wait! The insights you’ve acquired could offer a strategic advantage to your company so, why would you share them? Even if you are thinking that you should keep what you’ve gathered to yourself, I encourage you to share because AR is more than just another new technology offering you or your group a competitive advantage. This is going to be a major crowd-sourced, multi-year project. When more people are looking into AR technology, it will improve faster than when only a few are focusing on and investing in it in isolation.

Once AR is good enough to be used weekly (or daily) in more than one use case, it is going to push operational performance to new levels. Then you will be able to use it to full advantage.

AR may become as transformational to your company and industry as the Web and mobile devices during your professional career. But it requires more than one or two examples and adopters in an industry. Reaching a threshold level of adoption in your industry will be necessary. And, to begin meaningful adoption there need to be a few experts. We need people like you to translate the theory and potential of AR in your industry to practice and reality.

I’ve found that I can translate for others what I’m observing by breaking it down into four interrelated topics: hardware, software, content and services. For over a decade I’ve used these four legs of the AR platform to organize projects, to review the history of AR and to capture current status.

In a series of AREA blog posts I am sharing developments I believe will be important in AR in 2017 using this simple framework.

Connecting the dots around us

One observer can’t see all the details of the entire AR landscape, certainly not in all industries where the technology will apply. Fortunately, the AREA is a network of very bright minds that are also looking forward as well as in other directions, at the same time.

Many AREA members are in the trenches of Augmented Reality take on a forward looking challenge when, at the end of each year, they begin preparing their forecast for the following year.

I hope that these posts will permit you to find your place, connect yourself and in your comments to these posts, you will compare and contrast what you’ve observed with my experience.

If we each take a few minutes, hours or a day in this last quarter of 2016 to connect our dots together we will all be better equipped to concretely plan for an exciting year ahead!

Next: What’s new for AR hardware in 2017?

The post The AR Market in 2017, Part 1: Connect the Dots appeared first on AREA.

It’s not all that different for AR. Once an Augmented Reality project’s use case is clear, the experiences come about through an authoring process that resembles that of preparing and publishing content for the Web.

Figure 1. An AR authoring system combines trackables (created using features of the real world and a tracking library) with digital content that is encoded into presentation data and then assigned interactive functions (e.g., see more details, show relevant info, move and freeze in position, hide/close). The AR authoring system uses databases to store the scene elements – trackables, presentation data and interactions. (Source: PEREY Research & Consulting)

Today, Content Management Systems for the Web support the steps for page development with grace. Systems like WordPress and Drupal are so easy to use and commonplace that we hardly notice their existence.

In contrast, there are many AR authoring systems from which a developer can choose and none are as mature as CMS for the Web. The choice of tool and approach depends on the project requirements, skills of the developer and the resources available.

Define the AR Project Requirements

Before choosing an AR authoring system, the requirements must be clear. An AR experience design process should generate a storyboard and, from the storyboard, the following factors are defined:

• User settings (indoor, outdoor, noise levels, etc.)
• Need for a user management system to provide experience personalization or tracking
• Need for live communication with any remote experts during the experience
• Type of recognition and tracking required (marker, 3D, SLAM, etc.)
• Need to access device GPS and compass for geospatial context
• Preferred display device (smartphone, tablet, smart glasses or another type of HMD)
• Human interaction modalities (hands-free, touch, speech, gaze)

In addition to the above variables that can be deduced from the storyboard, there could be other factors to consider. For example, if the target device is connected by an IoT protocol or if there are any supplementary files (e.g., videos, PDFs, etc.), then these need to be provided to the developer as early as possible. The project manager should also specify the frequency and types of updates that may be required after the initial AR experience is introduced to users.

When these project requirements and parameters are defined, the developer can choose the tools best suited for the AR experience authoring.

Want to know more about your choices of authoring platforms? There’s more in the next post

The post When a Developer Needs to Author AR Experiences, Part 1 appeared first on AREA.

Choose a Development Environment

Someday, the choice of an AR development environment will be as easy as choosing a CMS for the Web or an engineering software package for generating 3D models. Today, it’s a lot more complicated for AR developers.

Most of the apps that have the ability to present AR experiences are created using a game development environment, such as Unity 3D. When the developer publishes an iOS, Windows 10 or Android app in Unity 3D, it is usually ready to load and will run using only local components (i.e., it contains the MAR Scene, all the media assets and the AR Execution Engine).

Although there’s a substantial learning curve with Unity, the developer community and the systems to support the community are very well developed. And, once using Unity, the developer is not limited to creating only those apps with AR features. The cost of the product for professional use is not insignificant but many are able to justify the investment.

An alternative to using a game development environment and AR plugin is to choose a purpose-built AR authoring platform. This is appropriate if the project has requirements that can’t be met with Unity 3D.

Though they are not widely known, there are over 25 software engineering platforms designed specifically for authoring AR experiences.

Table 1. Commercially Available AR Authoring Software Publishers and Solutions (Source: PEREY Research & Consulting).

The table above lists the platforms I identified in early 2016 as part of a research project. Please contact me directly if you would like to obtain more information about the study and the most current list of solutions.

Many of the AR authoring systems are very intuitive (featuring drag-and-drop actions and widgets presented through a Web-based interface), however most remain to be proven and their respective developer communities are relatively small.

Some developers of AR experiences won’t have to learn an entirely new platform because a few engineering software publishers have extended their platforms designed for other purposes to include authoring AR experiences as part of their larger workflow.

Or Choose a Programming Language

Finally, developers can write an AR execution engine and the components of the AR experience into an app “from scratch” in the programming language of their choice.

To take advantage of and optimize AR experiences for the best possible performance on a specific chip set or AR display, some developers use binary or hexadecimal instructions (e.g., C++) which the AR display device can run natively.

Many developers already using JavaScript are able to leverage their skills to access valuable resources such as WebGL, but creating an application in this language alone is slow and, depending on the platform, could fail to perform at the levels users expect.

To reduce some of the effort and build upon the work of other developers, Argon.js and AWE.js are Open Source JavaScript frameworks for adding Augmented Reality content to Web applications.

Results Will Depend on the Developer’s Training and Experience

In my experience, it’s difficult to draw a line between the selection of an AR authoring tool or approach and the quality or richness of the final AR application. The sophistication and quality of the AR experience in an app is a function of both the tools chosen and the skills of the engineers. When those behind the scenes (a) ensure the digital content is well suited to delivery in AR mode; (b) choose the components that match requirements; and (c) design the interactions well, a project will have the best possible impacts.

As with most things, the more experience the developer has with the components that the project requires, the better the outcomes will be. So, while the developer has the responsibility for choosing the best authoring tool, it is the AR project manager’s responsibility to choose the developer carefully.

The post When a Developer Needs to Author AR Experiences, Part 2 appeared first on AREA.

And new wearable segments are being defined. For example, Snap recently introduced its $130 Spectacles. 

Is this all good?

Thinly veiled behind the shiny new products is a vicious cycle.

The continual stream of announcements confirms for readers of this blog that the wearable AR display segment is still immature. This means that those customers with limited budgets seeking to select the best hands-free AR display for their projects in 2016 are likely to be disappointed when an update or new model appears, making the model they just brought in-house out of date. Risk-averse organizations may put their resources in another product category.

On the other side of this conceptual coin, the companies developing components and building integrated solutions for wearable AR must continue to invest heavily in new platforms. These investments are producing results — but without clear customer requirements, the “sweet spot” for which the products should aim is elusive. And when customers lack clear requirements, differentiating the latest offerings while avoiding hype is a continual challenge.

Breaking the cycle with specific requirements

When customers are able to prioritize their needs and provide specific product requirements and budgets, there’s hope of breaking this cycle.

The Electric Power Research Institute (EPRI) and PEREY Research & Consulting, both AREA Founding Sponsor members, have collaborated on the preparation of a new report entitled Program on Technology Innovation: State of the Art of Wearable Enterprise Augmented Reality Displays.

Targeting the buyers of wearable technology for use when performing AR-assisted tasks in utilities (and by extension, in other enterprise and industrial environments), the report seeks to demystify the key product features that can become differentiators for wearable AR solutions.

Based on these differentiators, the first multi-feature wearable AR display classification system emerges.

Source: Program on Technology Innovation: State of the Art of Wearable Enterprise Augmented Reality Displays. EPRI, Palo Alto, CA: 2016. 3002009258.

The report also discusses challenges to widespread wearable AR display adoption in technology, user experience, financial, and regulatory/policy domains.

Descriptions of a few “lighthouse” projects in utilities companies, logistics, manufacturing, and field service provide readers valuable insight into how early adopters are making the best of what is currently available.

This report is available for download at no charge as part of the EPRI Program on Technology Innovation.

If you have comments or feedback on the report, please do not hesitate to address them to the authors, Christine Perey and John Simmins.

The post New EPRI Report Offers Insights for Wearable AR Display Customers appeared first on AREA.

As a faculty-level unit within the University of Sheffield, the AMRC partners with industry to conduct advanced machining and materials research that brings high ROI. The AMRC with Boeing focuses on manufacturing in aerospace, automotive and other high-value industries while, in parallel, the Nuclear AMRC focuses on manufacturing innovation and supply chain development for the civil nuclear and energy sectors.

We asked the AMRC’s digital manufacturing assembly specialist, Chris Freeman, to shed light on the drivers for introducing Augmented Reality in manufacturing and the risks he sees when Augmented Reality is integrated into projects underway or planned at the AMRC.

How do you identify where and how Augmented Reality can offer value?

We don’t propose any new technology unless we believe that it can address a specific operator requirement.  When our research partners come to us with manufacturing challenges, we make sure we study those problems in relation to not just the senior directive, but also the hands-on operator who performs the job day in and day out. They know the problems best and, often they provide the key metric, or problem around which the application needs to focus. For example, a process may need to be completed with fewer concessions or at a greater rate, but the operator will want to focus on how it will make their own life easier. That operator-level buy-in is crucial to having a successful deployment. Their personal experience in processes needs to be considered just as much as the goals of those senior to them. By presenting solutions that serve requirements of both operator and managers, benefits like traceability or the elimination of errors will more likely be realized.

We might recommend exploring AR as part of a solution when the key business challenge (or the opportunity to reduce costs) involves people interfacing with and using complex instructions or information in context.   

In each scenario we need to closely examine the whole process to help build a preliminary ROI model. We are always looking for robust business cases, where technology integration can deliver a step change because going to the effort to introduce something new for small incremental changes is not going to be justifiable. Even if it is innovative, the technology will not be adopted.

Existing data about operator performance is often not available but we may be able to collect indirect metrics or indicators of efficiency such as the number of rework orders or how much scrappage (waste) a company generates. The details may be extracted from the company’s manufacturing execution system (MES) or standard operating procedures (SOPs). These systems have the ability to gather a lot of detail for other purposes that we can use as part of a study to understand the business case.

Do you use or integrate with real time sensor networks or IoT in any part of your projects?

The vision of Industry 4.0 has always included a component of connected machines communicating with, and being controlled by, systems and humans in intuitive and low-risk fashion. We are being requested to do more projects with the Industrial Internet of Things but, at the moment, it’s still exploratory.

Augmented Reality is a great enabler for humans working with IoT but a great deal of potential value of IoT rests in the architecture and systems that sit behind it. Sensor networks are very powerful, especially when combined with real time Big Data analytics, and the use of Augmented Reality will enable new methods of data visualisation and human interaction.

How is data prepared for use in Augmented Reality experiences in an advanced manufacturing environment?

Ideally there’s no need to introduce a new data manipulation step between the source of the data and the AR experience user, but a lot depends on the use case requirements. We recommend that the AR experience system uses the raw data straight from its source, whenever possible, and not duplicate any existing functionality. We also recommend that the digital content be as close to its native format as possible.

The more complexity there is in data handling or mining, the less robust and less repeatable the process becomes. Translation and optimisation is often necessary but it’s not ideal. In situations where access to data sources is not available, any transformation processes need to be as automated as possible.

To determine how suitable a client’s existing data portfolio is we will always work with the customer’s existing data sources to prove out the process. Often there’s a learning process through which everyone goes, which with our help, allows them to understand what can be achieved with their existing data.

Risk management plays a large role in the process, too. When we’re planning an application, we’re always looking for the key challenges and risks. We examine potential issues and document these to ensure we understand the potential pitfalls.

What are the sources of risk (challenges) when using AR in the manufacturing environment and how do you address them?

With Augmented Reality there are many uncertainties about the technology itself, such as how it works in different environments. We have all sorts of challenges around natural and artificial lighting conditions, wireless network connectivity and many other factors that will impact user interactivity. Manufacturing environments have high levels of ambient noise due to industrial-scale machinery. This noise presents challenges with speech-based recognition interfaces. Selecting the right AR interaction mode for the right task is crucial.

We see rapid change in the features of hands-free displays. This raises uncertainty about how long one model will last before being superseded by a new one. Each change introduces new risks and costs. In order to lower the impacts of frequent model updates, it’s important to first implement a robust back end architecture.  Then, once that’s in place, the AR experience presentation hardware (wearables, tablets, phones, etc.) can be quickly removed and changed without the cost and delays of changing the underlying architecture.

In addition, there are risks associated with different recognition technologies. We have to evaluate image, bar code, natural feature recognition, SLAM and depth sensing with respect to the project goals and the environmental constraints. As integrators, we can also combine AR with well-established technologies such as geo-location sensing, RFID and Bluetooth.

Many of our partners are very security conscious and tightly regulated. Systems purely reliant on cloud-based architectures will not even be considered. Local networks are an option but still very much a problem when a number of organizations we work with don’t (or can’t) have Wi-Fi on the shop floor. This drives us to look at solutions that work entirely offline and then can connect with a data infrastructure after a shift or task is completed.

There are also project risks due to excessively high expectations. In other words, hype. We work with all stakeholders to make sure they are clear and realistic about their goals and match those to what the technology can do today.

What are people’s attitudes towards the adoption of AR and how do you manage those?

It varies highly. Most have done some basic user studies prior to beginning any investment in order to understand the potential for adoption of a new technology. However, the exact process investigated is unlikely to be specific to the one you’re working on. They will be keen to learn more but will want to see tangible metrics around value and ROI. At an operator level, they will probably have little awareness of the technology and so may be cautious about its use and how it will impact their day-to-day life. User engagement and trials are crucial in order to get buy-in at a shop floor level. The operators need to be involved to show that the new technology is helping the end user do their work. Then, the feeling is more open and likely to have positive support. If the presentation system is a hindrance in any way it will be discarded, hence the importance of engaging with all sections of the business.

We always work closely with our partners to educate both ourselves about their use cases, and them about any new technology components. Everyone must have an open mind about the opportunities that AR enables and, as we said, the many risks. We will continue to encourage our partners to use a progressive, value-driven approach to adoption of any new technology. And we look forward to working more with our AR technology provider network, including AREA members, to deliver solutions to address existing manufacturing challenges.

The post Interview with the AMRC: Augmented Reality in Advanced Manufacturing appeared first on AREA.