Modeling and Planning Under Uncertainty Using Deep Neural Networks

Artificial neural networks (ANNs) have been frequently used in industrial applications to model complex systems. However, using traditional ANNs for longterm planning tasks remains a challenge as they lack the capability to model uncertainty. Process noise and approximation errors cause ANN long-term estimations to deviate from the real behavior of the system. Unlike traditional ANNs, stochastic models provide a natural way to model uncertainty, providing estimations over a range of several possible outcomes. This paper introduces a stochastic modeling and planning approach using deep Bayesian neural networks (DBNNs). We use DBNNs to learn a stochastic model of the system dynamics. Planning is addressed as an open-loop trajectory optimization problem. We present two approaches for learning the dynamics: using single-step predictions and using multistep predictions. The advantages of the proposed methodology are as follows. First, accurate long-term estimations of the system state-trajectory probability distribution without the need for expert knowledge of the dynamics. Second, improved generalization and faster convergence rates in the trajectory optimization task when using multistep predictions to train the model. Third, viable for real-world applications since all expensive optimizations are executed offline while using a reasonable number of data samples. Testing is performed using challenging underactuated benchmark problems: the Cartpole and the Acrobot. The presented methodology successfully learns the swing-up maneuver using a relatively small number of iterations, with less than 125 sampled trajectories, and without any expert knowledge of the dynamics.