1

Factory layout Experience – Theorem Solutions

Optimize designs in immersive XR

The Factory Layout Experience enables a planning or layout engineer, working independently or with a group of colleagues, locally or in remote locations, to optimize Factory layouts through the immersive experience of eXtended Reality (XR) technologies. Seeing your data at full scale, in context, instantly enables you to see the clashes, access issues and missing items which a CAD screen cannot show.

On the shop floor there are literally 1000’s of pieces of equipment- much of it bought in and designed externally. Building designs may only exist as scans or in architectural CAD systems, and robot cells may be designed in specialist CAD systems. There will be libraries of hand tools, storage racks and stillage equipment designed in a range of CAD systems, and product data designed in house in mechanical CAD. To understand the factory and assess changes, all of that has to be put together to get a full picture of where a new line, robot cell or work station will fit.

A catalogue of 3D resources can leverage 2D Factory layouts by being snapped to these layouts to quickly realize a rich 3D layout. Advanced positioning makes it very easy to move, snap and align 3D data. Widely used plant and equipment is readily available, there is no need to design it from scratch for every new layout. Simplified layout tools enable you to position, align and snap layout objects quickly, which can be used by none CAD experts, enabling all stakeholders to be involved in the process, improving communication.

Testing Design and Operational Factors

Human centred operations can be analysed using mannequins that can be switched to match different characteristics. You can test design and operational aspects of a variety of human factors, to determine reachability, access and injury risk situations, ensuring compliance with safety and ergonomic standards.

It enables companies to avoid costly layout redesign by enabling all parties involved to review the layout collaboratively, make or recommend changes, and capture those decisions for later review by staff who could not attend the session.




Inside the AREA Requirements Committee with Brian Kononchik

Inside the AREA Requirements Committee with Brian Kononchik

The work of the AREA is largely driven by its member committees: Research, Interoperability & Standards, Safety, Human Factors, Requirements, Marketing, and Security. Each of these groups is focused on activities that contribute to the development of knowledge about the adoption of enterprise AR and the practical implementation of AR use cases. For AREA members, participation in one or more of the AREA committees is an opportunity to share expertise, interact with other experts, and make a meaningful impact on the future of enterprise AR.

 

This is the third in a series of blog articles exploring the committees and their work. Our guide to the AREA Requirements Committee was Brian Kononchik, the Committee’s chair and Director of Innovative Technologies at Boston Engineering.

 

AREA: Tell us how you got into enterprise AR.

 

Kononchik: About 12 years ago, I started my career working alongside a prominent advanced technology investor and visionary. Together, we worked as contractors and consultants to some big names in the consumer electronics and automotive space. During that time, I realized that the status quo is never good enough, there needs to be more. That’s when I shifted my focus to innovation instead of that status quo. That’s how I became an early adopter of VR technology, holographics, and eventually AR. After a while, I jumped to an engineering firm that did development work for Siemens, which is, of course, a very big name in manufacturing with equipment, PLCs, PLM, and CAD platforms.

 

While I was there, one of the largest submarine manufacturers in the world needed an innovation engineer on their advanced technology team. The assignment was to create an immersive VR experience that utilized existing CAD data to provide an experience where a user can enter a submarine, navigate freely, and plan for job assignments. That led to more projects and eventually including Augmented Reality answering the questions: How can we use AR to help shipbuilders? How can we merge AR and VR to help shipbuilders collaborate and work more effectively?

 

Then about five years ago, the need for AR at scale became more and more prominent. I then went to work for PTC as a Director of Product Management for the Spatial AR initiative working with big name manufacturers in the automotive, semi-conductor, and manufacturing equipment space. That experience eventually brought me over to Boston Engineering where I’m now leading the Industry 4.0, Innovative Technology initiatives. I’m all about giving people value, and enterprise AR is delivering on that promise.

 

AREA: For our readers who aren’t familiar with it, what does the AREA Requirements Committee do?

 

Kononchik: We’re working in collaboration with a lot of big organizations involved with the AREA to help define global standards for hardware and software. When I say hardware, I am talking about mobile devices like the iPhone or Galaxy s20, Assisted Reality devices like RealWear, and fully-immersive, head-mounted displays, like the Magic Leap and HoloLens 2. We’re trying to define a set of standards that people could build hardware against. Having universal standards will allow for increased technology adoption.

 

On the software side, we’re trying to do something similar. We’re trying to lay out a set of standards for people that want to go build AR enterprise applications. After all the requirements are finalized, the next big thing we’re going to do is build out an automated process to help someone understand the starting point for addressing their particular use case. So, you would input your industry and use case. Say you’re in the oil and gas industry and you have the challenge of individuals collaborating while they are not in the same location. That would be a remote assistance use case. You then need to input your environment. In this scenario, you specify that the work is being performed mainly outside. Those are your three starting variables: oil and gas, remote assistance, and outside. We’re working on an automated process that recommends options; the more details you provide, the better the recommendation. We’re not just trying to define requirements. We’re trying to define requirements for use cases and provide a way to streamline adoption within an organization.

 

AREA: It sounds like you’re working two sides of the equation here. You’re pushing for standards on one end and providing guidance to adopters on the other.

 

Kononchik: Correct. We have our set of standards being developed, and then we have the AREA Statement of Needs, or ASoN, tool. It’s designed to help others identify the AR setups related to their use cases. These setups can be actions taken right away to get AR implemented into their organizations.

 

AREA: On the standards side, are you looking at work that standards bodies are already doing and making recommendations about which standards to implement?

 

Kononchik: It’s a combination. For example, we break hardware down into many different categories: wearability, sensors, communications, audio, and so forth. For each category, you have a breakdown of different device types. For example, sensors. You have mobile, assisted, and fully immersive head-mounted devices (HMD). A requirement for a fully immersive HMD should have no fewer than two world cameras because that will help you compute your 3D world maps. A device should also have at least a single RGB camera because that will help with QR code recognition and remote assist scenarios. So, those are some of the hardware type standards being developed.

 

When it comes to safety, though, we reference the industry standards. There are already existing environmental standards, say regarding operational temperature range, that hardware manufacturers must follow. So, we just reference those standards that exist. And then, where standards don’t exist, we’re trying to collectively understand what can be done today and where the market is going to establish new standards which organizations do not necessarily follow today.

 

AREA: The ASoN tool has been available for a while now. What’s the latest on that?

 

Kononchik: It is currently running on an older platform, and we are in the process of upgrading the application and migrating the data over to a new platform. So, people can use it and benefit from it today, but we are looking to cleanse the tool eliminating some bloat that accumulated over time. It is accessible and fully operational today and should be used and benefited from.

 

AREA: Do you have a wish list of things you’d love to see the Requirements Committee get done in the near future?

 

Kononchik: Right now, I want to see us go through and overhaul the hardware and software requirements and collectively agree on what they should be. We’re in the process of that now. Beyond that, my goal is to make the ASoN tool everything it’s promising to be. I would love to see it work so that, if say I were an automotive industry member and I entered in a very few parameters, I would get a full spec readout of what I needed to do next and why I need to do it. It’s not there yet, but I would love to see that.

 

AREA: Is that doable in the near future?

 

Kononchik: I would say within the next year, probably yes. That granularity will be well along its way. We have a lot of great minds working with us in this Requirements Committee – a lot of industry experience and industry knowledge, and not just hardware building and software development-specific. We also have input from the community members who focus on consulting with organizations, and they really understand what customers are looking for. When we join all this knowledge together, it really comprises the three pillars of successful AR implementation and development.

 

AREA: What kind of people are you looking for to become part of the Requirements Committee? Are there certain skills you need or certain types of people that you’re hoping can join the group?

 

Kononchik: We’re looking for multiple types of skillsets. First off, we’re looking for members of the hardware and software communities – smaller startups, enterprises, and places with new innovative solutions. And within those sectors, we’re looking for product leaders. Leaders in product strategy, product management, as well as technical areas to help get a good understanding of the market demands. Really want those feasibility, viability, and desirability type people – so, your business leaders, your product leaders, your design leaders, your technical leaders from both hardware and software. And we’re looking for the “go doers” – the people that go out to a customer and work with them for a year to implement AR. We want to understand the pain points of implementation, adoption, and scalability.

 

The onset of the pandemic really escalated the adoption of AR, and a lot of companies are seeing challenges adopting and scaling AR. Those companies are making do with the pain of adoption. The Requirements Committee can increase the visibility of the challenges adopters are facing and make those challenges more visible to the hardware creators and software development companies. Having visibility like this allows these organization to develop a strategy that fits market demands, satisfying their business, but more importantly, the customer.

 

If you’re an AREA member and would like more information about joining the AREA Requirements Committee, contact Brian Kononchik or AREA Executive Director Mark Sage. If you’re not yet an AREA member but care about ensuring safety in enterprise AR, please consider joining; you can find member information here.

 

 




Multi-layered Approach to Public Asset Utilization in Smart Cities

There is a wide range of assets in urban environments: public spaces, buildings, concentrations of demography, utilities, history, movement of people and objects and so on. These assets carry significant value for all those interacting with them but the nature and manifestation of the value depends on the context and the use case.

The nature of value associated with the same asset depends on the nature of the interaction: occupancy for a realtor, history for a tourist, compliance for the Code Enforcement employee and so on – all built around the same digital twin of the host city.

Following proper authorization, head mounted displays or handheld devices can unlock the value contained in an asset, guiding and enhancing the interactions of a user with a city and its inhabitants.

This research topic also includes architecting a multipurpose Digital Twin of a set of assets, associating them with various layers of value and examining modalities of consumption from a human factors perspective.

Stakeholders

Elected and professional urban leaders: Mayors, CIOs and CISOs of urban infrastructure, Independent Software Vendors, geospatial infrastructure providers, points of interest publishers

Possible Methodologies

This research will require scanning and associating data with urban assets and developing methodologies to test different use cases and scenarios. User satisfaction and productivity studies in field trials will contribute to development of best practices for specific industries.

Research Program

This topic is a good fit with most topics focused on Smart Cities and long-range outdoor positioning.

Miscellaneous Notes

The UN has published reports about urbanization and the challenges it raises for those managing urban data.

Keywords

Public asset management, public services, scanning, smart cities, urbanization, intelligent buildings, urban growth, town and country planning, urban planning, digital twin, smart cities, street lighting

Research Agenda Categories

Technology, Business

Expected Impact Timeframe

Long

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

More publications can be explored using the AREA FindAR research tool.

Author

Peter Orban

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard




Informing AR Users About Hazards in Proximity

In many industries, workplaces contain a plethora of hazards. When known or anticipated, hazard management protocols reduce the risks associated with a user’s encountering a hazard when performing tasks or fulfilling a work order. However, there are also hazards which, even if made aware of them, the user is untrained to treat or has insufficient time to avoid or deescalate. Alerts can provide the user time to react. On the other hand, there may be hazards that do not require any specific user actions.

Through data collected by user location sensing technologies on devices, and potentially on users’ PPE, as well as maps of known hazards, data generated from sensors on stationary or moving machines, and other methods, artificial intelligence algorithms could be used to continuously maintain and monitor a dynamic 3D map of hazards in a user’s proximity. The user may be provided the hazard proximity map at intervals or request to visualize hazards in proximity. When the user reaches conditions with respect to the hazard that suggests appropriate actions are needed, an alert on an AR device can spatially anchor the source of risk or hazard in the user’s perception (see Automated Alert to Dangerous Settings [[ra-Salert5-dangerosity]]). If and when needed, guidance for risk mitigation can be provided.

Stakeholders

Safety managers, workplace designers, risk managers

Possible Methodologies

This research can include studying appropriate definitions of proximity and risk in diverse industries and workplaces and/or using existing risk management tools, capturing data sets and training algorithms for types of hazards and testing reliability of AI in diverse conditions. The study of user interface and user experience for hazard notification systems will contribute to this field. Further, user studies will be required to measure cognitive load and user responsiveness to notifications of hazardous or potentially hazardous circumstances.

Research Program

The scope of this research can span many industries and workplaces. It could be tailored to any industry in which AR is introduced and demonstrated with many workplace use cases. It is closely related to other proposed topic concerned with Automated Alert to Dangerous Settings, and a topic focusing on dangers due to chemical or radiation in employee vicinity. It could be combined with research on visualization of IoT data streams, 3D maps of known risk and other safety management programs. Further, it also can include or be an extension of numerous 3D user interface and user experience topics.

Miscellaneous Notes

This peer-reviewed article was published in December 2015 entitled “Proximity hazard indicator for workers-on-foot near-miss interactions with construction equipment and geo-referenced hazard areas” pertains to the topic of this research.

Keywords

Hazard detection, hazard management protocol, hazard warning, location-detection, 3D spatial mapping, artificial intelligence, user interface, user experience, risk assessment, risk management, situational awareness, occupational risks, risk assessment, risk perception, accidents, occupational health, occupational safety, safety, health and safety, health hazards, safety devices, safety factor, safety systems, fault detection, monitoring, system monitoring

Research Agenda Categories

Industries, Technology, Business

Expected Impact Timeframe

Near

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

More publications can be explored using the AREA FindAR research tool.

Author

Christine Perey

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard




Biometric Identification of Wearable Enterprise AR Device Users

New AR display devices encounter significant resistance from enterprise IT teams who consider the new hardware platforms increase security threats, consequently, increasing the need for an elevated security posture.

Driven by a human-centric approach, a critical step in ensuring compliance with existing security policies and systems is to balance the security with accurate and rapid user authentication and ultra-low-friction user input.

Biometric identification methods, ranging from palm-prints, voice-prints, iris scanning, gait to heartbeat detection offer a plethora of opportunities for identification of wearable enterprise AR device users before providing access to enterprise work orders and data.

This research topic compares different modalities of biometric identification and classifies them based on accuracy, cost and ease-of-use.

Stakeholders

All stakeholder in corporate security organizations but primarily CISOs, CIOs, IT and security managers. On the vendor side, OEMs, solutions providers, system integrators and independent software vendors will be impacted by this research.

Possible Methodologies

This research will require rigorous laboratory tests, deployed via multiple cells of different modalities. This will be followed by human factors and security research, culminating in field trials. Once baselines are available and validated, best practices can be established.

Research Program

This topic is closely related to another proposed AREA Research Agenda topic on cleaning and authenticating multi-user devices end user [ra-Tsecurity5-multiuserdisplays]. The topics could be combined with other AR security topics to develop a broader research program. In addition, the topic could be expanded to use the sensors on devices of other AR users in a workplace to confirm user identities.

Miscellaneous Notes

The field of biometric authentication in cybersecurity is vast and there are many highly reputable research centers that could contribute to this research. Hundreds of publications appear each year in journals and proceedings. This paper describes results of studies to connect AR users with sensitive personal information derived from on-line platforms and use of these data to predict AR user interests and preferences. In the https://dl.acm.org/doi/proceedings/10.1145/3457339[proceedings of the 7th ACM on Cyber-Physical System Security Workshop] (May 2021) an article compiles recently published work on this topic and describes MoveAR. The goal of MoveAR is to distinguish between a legitimate user and potential adversaries based on the signatures detected by the on-device sensors as the user interacts with an augmented reality environment.

Keywords

Biometric, palm print, voice print, gait, retina scanning, iris scanning, heartbeat detection, skin conductivity, access control, data protection, security systems, authentication, message authentication, authorization, data security, access protocols

Research Agenda Categories

Displays, End User and User Experience, Technology

Expected Impact Timeframe

Medium

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

More publications can be explored using the AREA FindAR research tool.

Author

Peter Orban

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard