What is eXtended Reality and how does it help workers?

What is eXtended Reality and how does it help workers?

Through all the industrial revolutions, tools and machines have been central to workers’ realities. But it is only recently that large portions of a worker’s reality could be digitized with IoT devices and approaches. In 2015, Henning Kagermann, former CEO of SAP AG, argued that this “digitization—the continuing convergence of the real and the virtual worlds will be the main driver of innovation and change in all sectors of our economy.”  This simple act of creating digital streams produces information that can be expressed in many different ways, on many different types of materials, and in many different systems. This article argues that modern reality presentation technologies are compelling mediums for the expression of digital IoT streams.

Such reality presentation technologies include the eXtended Reality (XR) family of technologies. Augmented Reality (AR), Mixed Reality (MR) and Virtual Reality (VR) – as well as more mature and accepted technologies such as smart phones, tablets, and PC flat screens. When combined with IoT, analytics and artificial intelligence, applications can be created that can aid workers by making their realities more intelligent.

An intelligent reality is defined here as a technologically enhanced reality that aids human cognitive performance and judgement. As compared to the base reality, an intelligent reality can have much greater dimensionality, reduced occlusion, transcendence of distance, better guidance and improved communication with other actors. This definition deliberately does not exclude non-physical realities in domains such as finance and cybersecurity, but the focus of this article is on intelligent realities based on physical realities and fed by IoT.

Consider a technician looking at a machine while wearing an AR Head Mounted Display (HMD) can see both the service history and prediction of future failures. This gives the worker a view on the fourth dimension of time, both backwards and forwards. Instead of having to take the machine apart, the worker can see an IoT driven mixed reality rendering projected on the outside casing. By just glancing away from the machine, he can see a virtual rendering of the operations of the same type of machine at a distant location. Then, he can interface with both artificial and human remote experts about next steps, which could include the expert driving virtual overlays of his view. As he decides on next steps, he can communicate with appropriate management systems through that same HMD without having to pull out a phone or laptop. As a wearable computer, the HMD brings distant resources in to the worker’s operational reality.

An intelligent reality may be proximate to a worker, like a machine on a factory floor. Or that factory might be half way around the world and understood by the user through 3D modeling of the factory. AR or VR headsets may be involved, but do not have to be – smart phone screens or flat screens on a desktop may be a better option. The worker may be mobile and use an AR head mounted display or smart phone, or the worker may be stationed in a command center at the company headquarters. They may be observing a reality in real time, or they may be performing data-driven review of an event that occurred in the past. In all cases, though, the context dominates – both visually and in the design of the presentation.

Intelligent reality can be achieved today with off-the-shelf technologies spanning IoT, analytics, XR technologies, and more traditional user interface technologies. This new paradigm is introduced here to help decision makers and architects navigate the expansive terrain of technologies that can enable intelligent realities for workers. First, the XR space is overviewed along with more traditional mobile and desktop flat screens. This leads to the consideration of intelligent reality architecture and the development of intelligent reality applications. From there, specific use cases are proposed that exercise combinations of reality presentation technologies, IoT and AI.

This is an excerpt from the “Intelligent Realities For Workers Using Augmented Reality, Virtual Reality and Beyond” white paper written by Michael D. Thomas, Senior Systems Architect SAS, for The Industrial Internet Consortium.

Read here the latest edition of the Journal of Innovation, published yesterday by The Industrial Internet Consortium.