Stretchable e-skin and transformer enable high-resolution morphological reconstruction for soft robots

Developing proprioception systems for flexible structures such as soft robots is a challenge. Hu et al. report a stretchable e-skin for soft robot proprioception. Combined with deep learning, the e-skin enables high-resolution 3D geometry reconstruction of the soft robot and can be applied in many scenarios, such as human-robot interaction. Many robotic tasks require knowledge of the exact 3D robot geometry. However, this remains extremely challenging in soft robotics because of the infinite degrees of freedom of soft bodies deriving from their continuum characteristics. Previous studies have achieved only low proprioceptive geometry resolution (PGR), thus suffering from loss of geometric details (for example, local deformation and surface information) and limited applicability. Here we report an intelligent stretchable capacitive e-skin to endow soft robots with high PGR (3,900) bodily awareness. We demonstrate that the proposed e-skin can finely capture a wide range of complex 3D deformations across the entire soft body through multi-position capacitance measurements. The e-skin signals can be directly translated to high-density point clouds portraying the complete geometry via a deep architecture based on transformer. This high PGR proprioception system providing millimetre-scale, local and global geometry reconstruction (2.322 +/- 0.687 mm error on a 20 x 20 x 200 mm soft manipulator) can assist in solving fundamental problems in soft robotics, such as precise closed-loop control and digital twin modelling.

Automated summary

Researchers have developed a stretchable e-skin combined with deep learning for high-resolution 3D geometry reconstruction of soft robots, which can be used in various scenarios like human-robot interaction.