Fuzzy Deep Deterministic Policy Gradient-Based Motion Controller for Humanoid Robot

In conventional robot arm control, inverse kinematics (IK) is used as the basis for computing arm joint angles. However, IK can be used to compute joint angles only after the terminal point has been reached, and it cannot optimize arm movements. Furthermore, the singularity problem is sometimes encountered when using IK. For example, if a robot arm in motion passes through a singularity, the next step's movement is incomputable, which results in errors. Therefore, this study did not use IK for computing the joint angles of humanoid robot arms. Instead, this paper proposes a motion controller based on machine learning and fuzzy logic for the aforementioned purpose. Conventional reinforcement learning can provide satisfactory results for a single state but cannot be used to perform optimized calculations for infinite states. To solve this problem, this study used the deep deterministic policy gradient (DDPG) algorithm and allowed a humanoid robot to self-learn and autonomously plan the movement of and joint angles in its arm. A state and its action can be calculated in a hyperplane by using the developed neural network. A continuous mapping relationship exists between the state and its action in this hyperplane. Thus, the humanoid robot obtained optimal learning experiences in multiple self-learning processes. Finally, the concept was incorporated into a visual feedback system to achieve object grasping by the humanoid robot. The humanoid robot exhibited satisfactory learning outcomes-as well as satisfactory motion control performance-in experiments when combining fuzzy logic with the DDPG algorithm.

Automated summary

This paper proposes a motion controller based on machine learning and fuzzy logic for humanoid robot arm control. By using the deep deterministic policy gradient (DDPG) algorithm, the humanoid robot can self-learn and autonomously plan the movement and joint angles of its arm. Experimental results show satisfactory learning outcomes and motion control performance when combining fuzzy logic with the DDPG algorithm.