A First Evaluation of a Multi-Modal Learning System to Control Surgical Assistant Robots via Action Segmentation

The next stage for robotics development is to introduce autonomy and cooperation with human agents in tasks that require high levels of precision and/or that exert considerable physical strain. To guarantee the highest possible safety standards, the best approach is to devise a deterministic automaton that performs identically for each operation. Clearly, such approach inevitably fails to adapt itself to changing environments or different human companions. In a surgical scenario, the highest variability happens for the timing of different actions performed within the same phases. This paper presents a cognitive control architecture that uses a multi-modal neural network trained on a cooperative task performed by human surgeons and produces an action segmentation that provides the required timing for actions while maintaining full phase execution control via a deterministic Supervisory Controller and full execution safety by a velocity-constrained Model-Predictive Controller.

Automated summary

The future development of robotics lies in achieving autonomy and cooperation with human agents, especially in tasks requiring high precision and significant physical strain. To ensure the highest safety standards, a deterministic automaton is necessary, although it may not be adaptable to changing environments. This paper introduces a cognitive control architecture utilizing a multi-modal neural network trained on tasks performed by human surgeons, providing necessary action timing control.