SPIRIT ROTIE

ROTIE is a distributed and fully containerization-compatible platform conceived to allow robot operators to monitor and drive their mobile robots or manipulators through immersive interfaces working in bi-directional mode. The framework will offer APIs for advanced Human Interaction Devices that are used to facilitate user-intention-based robot control. And, on the other way round, the user will be virtually tele-transported to the robot environment through the use and fusion of sensorial data including principally 2D/3D images and spatial sound. Besides, the platform is designed to allow also the development of digital twins that can extend the performance of robot operations for pre-operation planning and post-operation analysis.

This platform consists of dedicated modules for each layer of human-machine interaction: HID API, XR Service, Robot Motion and Sensor APIs, Robot Operation Engine, Scene Reconstruction Engine and Virtual Scene Rendering Engine. It is conceived as a block-structure, which will allow different types of robots and HID devices to be integrated easily with the platform interfaces. In combination with the ROTIE services, it will contribute to the standardization of robot tele-operation interfaces.

Considering the restrictions and possible dangers of remote control of robots, security and data access control layers must be included. In addition, low latency in communications is necessary to allow reliable control of the robots. The ROTIE platform would be integrated with the different layers of the SPIRIT architecture at different levels, in order to test the reliability and precision of control using immersive devices.

Objectives:

ROTIE project aims to transform human-to-machine and machine-to-machine interactions by developing new HID (Human Interface Devices) to remotely operate robots, recreating the physical space where the robot is in, so the remote worker is perfectly aware of the environment and can control and operate the robot to perform specific tasks in this environment.

The potential benefits of operating robots in hazardous, dangerous or remote environments are clear, including workers’ safety, precision and consistency and cost-effectiveness. Combining robots and immersive technologies seems the step forward to transform human-to-machine interactions, both in physical and virtual environments.

To achieve this goal, four specific objectives were defined:

• Data acquisition. Using sensors from the robot (LiDAR, cameras and microphones) data will be acquired and pre-processed, enabling to send these data to a cloud server.
• Creation of immersive environments with HID. The working space will be re-created in a desktop application or mobile / wearable application, giving the operator information about this environment and allowing to run the robot with accuracy and adapting to the current conditions in the real world.
• Integration with SPIRIT framework. This environment will be seamlessly integrated into the SPIRIT platform, providing the system with intuitive human-centred interfaces that use the standards for data communication included in the SPIRIT framework.
• Testing and validation of the integrated system, both in ITG facilities and in the SPIRIT testbed (Deutsche Telekom). The aim of the application is to take advantage of the features given by the SPIRIT platform, mainly regarding communication availability and latency, data security and cloud processing capabilities