Introducing separable utility regions in a motivational engine for cognitive developmental robotics

Cognitive Developmental Robotics relies on lifelong open-ended learning processes, where mechanisms are needed to allow the robot to self-discover and self-select goals as well as to self-define its state space evaluation with regards to them. Thus, this paper addresses the problem of finding and using goals in continuous state spaces and automatically obtaining sub-goal hierarchies that allow autonomous development. In particular, the main purpose of this paper is to propose a new approach to the creation of utility models based on the concept of separable utility regions (SURs), which reduce the complexity of standard value function like utility models. These regions exhibit a correlation between the expected utility and the response of one sensor of the robot. Once they are discovered, the evaluation of the candidate states is only based on the changes of one sensor, which provides a strong independence from noise or dynamism in the utility models. A non-static variation of the classical collect-a-ball scenario and a robot gathering problem were used to test this approach in simulation and on real robots in order to identify goals and sub-goals in an autonomous way. The results confirm the good response of the method as a highly promising approach towards autonomous learning of continuous domains in cognitive robotics.