APPROXIMATING OPTIMAL FEEDBACK CONTROLLERS OF FINITE HORIZON CONTROL PROBLEMS USING HIERARCHICAL TENSOR FORMATS

Controlling systems of ordinary differential equations is ubiquitous in science and en-gineering. For finding an optimal feedback controller, the value function and associated fundamental equations such as the Bellman equation and the Hamilton-Jacobi-Bellman equation are essential. The numerical treatment of these equations poses formidable challenges due to their non-linearity and their (possibly) high dimensionality. In this paper we consider a finite horizon control sys-tem with associated Bellman equation. After a time discretization, we obtain a sequence of short time horizon problems which we call local optimal control problems. For solving the local optimal control problems we apply two different methods; one is the well-known policy iteration, where a fixed-point iteration is required for every time step. The other algorithm borrows ideas from model predictive control by solving the local optimal control problem via open-loop control methods on a short time horizon, allowing us to replace the fixed-point iteration by an adjoint method. For high -dimensional systems we apply low-rank hierarchical tensor product approximation/tree-based tensor formats, in particular tensor trains and multipolynomials, together with high-dimensional quadra-ture, e.g., Monte Carlo. We prove a linear error propagation with respect to the time discretization and give numerical evidence by controlling a diffusion equation with an unstable reaction term and an Allen--Cahn equation.

Automated summary

This paper discusses the finite horizon control problem in ordinary differential equation systems, and presents two different methods for solving it: policy iteration and model predictive control. For high-dimensional systems, low-rank tensor approximation and high-dimensional quadrature methods are used for numerical solution, and the effectiveness of the methods is verified through examples.