Multimodal Estimation of Endpoint Force During Quasi-Dynamic and Dynamic Muscle Contractions Using Deep Learning

Accurate force/torque estimation is essential for applications such as powered exoskeletons, robotics, and rehabilitation. However, force/torque estimation under dynamic conditions is challenging due to changing joint angles, force levels, muscle lengths, and movement speeds. We propose a novel method to accurately model the generated force under isotonic, isokinetic (quasi-dynamic), and fully dynamic conditions. Our solution uses a deep multimodal convolutional neural network (CNN) to learn from multimodal electromyogram-inertial measurement unit (EMG-IMU) data and estimate the generated force for elbow flexion and extension, for both intra- and intersubject schemes. The proposed deep multimodal CNN extracts representations from EMG (in time and frequency domains) and IMU (in time domain) and aggregates them to obtain an effective embedding for force estimation. We describe a new dataset containing EMG, IMU, and output force data, collected under a number of different experimental conditions, and use this dataset to evaluate our proposed method. The results show the robustness of our approach in comparison to other baseline methods and those in the literature, in different experimental setups and validation schemes. The obtained R-2 values are 0.91 +/- 0.034, 0.87 +/- 0.041, and 0.81 +/- 0.037 for the intrasubject and 0.81 +/- 0.048, 0.64 +/- 0.037, and 0.59 +/- 0.042 for the intersubject scheme, during isotonic, isokinetic, and dynamic contractions, respectively. In addition, our results indicate that force estimation improves significantly when the kinematic information (IMU data) is included. Average improvements of 13.95%, 118.18%, and 50.0% (intrasubject) and 28.98%, 41.18%, and 137.93% (intersubject) for isotonic, isokinetic, and dynamic contractions, respectively, are achieved.

Automated summary

This study proposes a novel approach for force/torque estimation using a deep multimodal convolutional neural network (CNN) with electromyogram-inertial measurement unit (EMG-IMU) data. The results show the robustness of the method and significant improvement in force estimation when incorporating kinematic information.