Introduction

Movicon.NExT has been designed to allow users to take the advantage of the most advanced Machine Learning technologies aimed at detecting and recognizing objects in the field to immediately visualize their dynamic real-time information for operator scrutiny.

This technology allows operators to use their mobile device’s integrated video camera or augmented reality devices to combine cognitive system information with the Movicon.NExT project by using any HTML5 browser with the WebHMI technology. This document describes the technologies on which the platform is based and how to use and configure them.

Cognitive Augmented Reality Applications with Movicon.NExT

The scope of this solution, which is integrated in Movicon.NExT, is to facilitate the creation of HMI ystems with the capability to use the camera eye lens to recognize previously learned elements or omponents in the plant. Due to the real-time recognition of objects in the surroundingenvironment, operators can automatically receive the relative HMI real-time information through utomatic pop-up screens previously defined in the project in relation to the targeted context. The MI screens can in turn contain dynamic project information by displaying variables in animated screen format and allow the use of interactive objects to invoke commands.

For example:

1. Once the browser has been opened and the video camera activated, the operator will be able to frame their surrounding environment .
2. A scroll slider has been provided for all desktop and Android devices with a Google Chrome browser to apply a zoom factor to the camera.  Other devices, such as Apple or other browsers do not support this feature.
3. The system will analyze the streamed video in order to recognize the objects it has been trained to identify and indicate them on screen enclosed within a boundary box;
4. Once the target object has been recognized and tracked, a pop-up window will show with its corresponding information. This gives the operator direct access to all the information typical of an HMI, such as displayed digital data, alarms and historical information, along with the ability to execute operational commands.

The difference between Augmented Reality and Mixed Reality

Augmented Reality refers to the visualization of real-world images captured by the camera in devices that are overlaid with additional dynamic information processed by the Artificial Intelligence systems. Augmented Reality is supported by devices equipped with rear cameras such as tablets and smartphones. Mixed Reality refers to the management of devices that offer a more immersive experience allowing operators to interact with and manipulate both physical and virtual items and environments using wearable devices such as the Microsoft HoloLens with next-generation sensing and imaging technologies. These holographic devices display additional dynamic information through the holographic visualization of images displayed in the surrounding space using appropriated holographic displays. From a functional standpoint, there are no differences in use between the two. However, from a technical standpoint, the process technologies are slightly different, as illustrated in the following diagram: