Towards commissioning, resilience and added value of Augmented Reality in robotics: Overcoming technical obstacles to industrial applicability

Augmented Reality (AR) has the potential for facilitating the interaction with robots by enhancing the operator's spatial understanding as well as providing further cognitive support, e.g. in order to make manual programming processes more efficient and provide on-site simulation. However, we consider major issues that prevent a widespread use of AR towards robotics in industries. Initially, the commissioning of AR devices for robotic applications demands spatial referencing of robots and AR devices. Fiducial markers are a popular artificial aid, but hard to implement in industrial use cases because of additional efforts and a lack of robustness. A further bottleneck for developing AR applications in robotics is the restricted technical and ergonomic maturity of AR devices, e.g. limited local computation and short product life-cycles. Furthermore, benefit through AR in comparison to classic online and offline programming techniques is still unclear for most applicators. In this paper, we present approaches towards a distributed, hardware-agnostic microservice architecture with standard interfaces for an interoperable usage of AR in robotics. Furthermore, we present marker-less pose estimation methods for articulated robot arms as well as mobile robots based on RGB and depth images that serve automatic referencing. Finally, we reveal further application potential of AR in robotics in terms of combining advantages from online and offline programming.

Automated summary

Augmented Reality (AR) has the potential to enhance interaction with robots but faces challenges in terms of spatial referencing, technical limitations of AR devices, and uncertainty in comparison to traditional programming techniques.Developing a hardware-agnostic microservice architecture and marker-less pose estimation methods are proposed as solutions to these challenges.