Hierarchical POMDP planning for object manipulation in clutter

Object manipulation planning in clutter suffers from perception uncertainties due to occlusion, as well as action constraints required by collision avoidance. Partially observable Markov decision process (POMDP) provides a general model for planning under uncertainties. But a manipulation task usually have a large action space, which not only makes task planning intractable but also brings significant motion planning effort to check action feasibility. In this work, a new kind of hierarchical POMDP is presented for object manipulation tasks, in which a brief abstract POMDP is extracted and utilized together with the original POMDP. And a hierarchical belief tree search algorithm is proposed for efficient online planning, which constructs fewer belief nodes by building part of the tree with the abstract POMDP and invokes motion planning fewer times by determining action feasibility with observation function of the abstract POMDP. A learning mechanism is also designed in case there are unknown probabilities in transition and observation functions. This planning framework is demonstrated with an object fetching task and the performance is empirically validated by simulations and experiments. (C) 2021 Elsevier B.V. All rights reserved.

Automated summary

This work introduces a new hierarchical POMDP framework for object manipulation tasks, which improves planning efficiency by extracting a brief abstract POMDP and includes a learning mechanism for unknown probabilities. The framework is demonstrated with an object fetching task and validated empirically through simulations and experiments.