Providing Automatic Feedback to Trainees after Automatic Evaluation

Learning how to perform precise and controlled gestures is difficult, especially when feedback about made errors is sparse. Therefore, some works try to facilitate learning by providing virtual coaches. Most of them propose to automatically score task quality. But simply assessing quality through a score is not enough. Indeed, it is essential to provide explanations on assigned scores just like experts do when supervising trainees. However when quality assessment is done automatically, such explanations are rare and computing an automatic feedback is complex. In this work, we propose to address this problem by providing an automatic feedback based on neural network explanation. Contrary to previous state of the art methods, which are focused on neural networks explicability for classification tasks, we want to explain network decision on a regression problem (quality score prediction). Thus, we propose to use gradient-based approaches and adapt them to a regression task. Moreover, to address the problem of noise present in sensitivity maps, we propose a solution that leads to more robust gradients. To test our approach, since automatic quality assessment datasets do not contain ground truth about errors position and amplitude, a synthetic dataset representing a simple temporal task has been created, with its associated ground truth. Once the method has been validated on this synthetic dataset, we apply it on real data composed of robotic surgical gestures.

Automated summary

This study aims to address the issue of quality assessment by providing automatic feedback based on neural network explanation, focusing on explaining network decisions for quality score prediction. By using gradient-based approaches and adjusting them for better robustness, the method aims to provide a more accurate assessment.