Augmented Reality in IoT

Augmented Reality in IoT

The use of augmented reality in IoT

An interview with Ian Kanski, CTO of UrsaLeo

The use of augmented reality (AR) in IoT and industrial applications is growing by leaps and bounds and has become tantamount in Industry 4.0 concepts. In fact, 28 percent of those surveyed in a recent PwC study reported that they have implemented, piloted, or planned to implement AR and/or virtual reality (VR) technology.

AR enables workers to access digital information and combine it with the physical world. When used with a photorealistic 3D digital twin, it becomes even more powerful, allowing the on-site workforce to interact with offsite experts when a challenge arises or questions on how to operate or fix something comes up.

We sat down with Ian Kanski, CTO of UrsaLeo, to talk about the benefits AR brings to the industrial space and how it is being applied to 3D technology. His answers were enlightening and helped to showcase what the future holds for AR and how it will further empower your workforce while also saving time and money.

Q: In its simplest form, how does AR work and how do you see it being applied in the industrial space?

Simply put, AR adds digital embellishment to a user’s local environment by using the physical world as a canvas. AR combines our natural visual context with image enhancements to create increased levels of cognition and understanding not achievable with the naked eye. In the industrial space, AR enables a user to consume more pertinent information in a concentrated collection and derive situational context that would normally be lost in disparate context streams.

Consider a typical monitored piece of machinery/equipment. Sensors are situated at critical areas, sampling data points that are sent to an IoT dashboard. There are piles of technical manuals and equipment specifications, sometimes integrated into the dashboard for access, other times not. Maintenance records are often in separate systems from monitoring solutions or not correlated in meaningful ways. Coupled with an inventory of what could be hundreds if not thousands of industrial components in a given environment, gaining a unified and actionable context of a given piece of equipment during troubleshooting is a challenge to say the least.

With AR, the above data sources can be applied in a targeted and focused manner, leveraging the intimate context of being adjacent to the physical piece of equipment. For instance, sensors can be rendered as an overlay on the equipment for a holistic view of readings across the asset. AR users can view live and historical data with operational context derived from seeing the actual physical location of the sensor on the equipment, in the system as a whole. Dynamic and empowering workflows can be created with AR where the user can investigate sensor alerts, identify suspect asset components upstream and downstream of the physical sensor location, access service notes and maintenance records all consolidated and rendered interactively on the equipment.

AR users can render a local digital twin of the equipment, move and manipulate the twin for critical physical context of a large, often immovable or difficult to reach piece of industrial machinery. The AR user can even select components of the equipment, pull a digital copy out and break it apart as a maintenance reference in AR.

Q: Can you talk about specific types of industries that will benefit greatly from the use of AR?

Any industry that can benefit from visual context, either as an internal function or as part of their product (directly or indirectly) can benefit from the use of augmented reality. Rather than thinking of AR applicability to a given industry, it’s better to consider how it enables core competencies that reside in multiples across all industries. Sales and marketing, training and education, operational monitoring, maintenance operations, research and development, all of these essential functions can benefit greatly from AR enablement. More about the key role of Augmented Reality in Industry 4.0 for Manufacturing.

As an example, I just recently bought a tent for family camping. I was searching through a large segment of the manufacturers trying to find one that had good headroom (we are a tall family). I must have looked at a hundred online with a measuring tape in hand. The one I decided to go with had an AR rendering component to their product website. I was able to “pitch” the tent in my living room and literally walk inside. It had great head room. That function got them a sale that day.  My point, aside from the necessity of good head room in a family tent, is that AR is a technical force multiplier and widely applicable.

Q: What are the barriers to entry in getting users up and running with AR applications in IoT?

Like any burgeoning technology there are always technical hurdles to negotiate during implementation. That said, AR frameworks have improved by leaps and bounds providing streamlined integration enablement, significantly lowering the barrier to entry from a development point of view.

The proliferation of smartphones, as the most basic entry point from a hardware perspective, coupled with WebAR have made creating impactful, value-added AR integrations widely attainable. For more advanced integrations, like those applied in an industrial physical setting (AR overlay of a piece of equipment), AR alignment and accuracy can be a challenging problem to solve. The inherent natural variations and variables associated with one rendering environment to another represent a moving target. Still these challenges can be overcome through structured application of AR and developments in markerless AR, specifically simultaneous localization and mapping technology (SLAM), have opened the door to new possibilities.

Q: What’s next? What does the future hold for AR in the industrial space?

As augmented reality in IoT continues to evolve we will start to see companies combining disparate context streams with the physical world. This will enable their teams to work more efficiently. In addition, customers will be able to experience products more contextually (e.g., the AR tent) while helping to drive sales and conversions.

We will also begin to see the adoption of dedicated mixed reality (MR) headsets, like the Hololens 2. This will become the future for AR consumption in industrial settings. The hands-free design will eliminate the need to hold a smartphone or tablet up to a piece of equipment or physical object. It will enable hand tracking and rendering manipulation that was traditionally only found in VR formats. The hands-free design will also be safer while offering a more functional user experience in physical settings.