Nonlinear-nonquadratic optimal and inverse optimal control for discrete-time stochastic dynamical systems

In this article, we investigate the role of Lyapunov functions in evaluating nonlinear-nonquadratic cost functionals for Ito-type nonlinear stochastic difference equations. Specifically, it is shown that the cost functional can be evaluated in closed-form as long as the cost functional is related in a specific way to an underlying Lyapunov function that guarantees asymptotic stability in probability. This result is then used to analyze discrete-time linear as well as nonlinear stochastic dynamical systems with polynomial and multilinear cost functionals. Furthermore, a stochastic optimal control framework is developed by exploiting connections between stochastic Lyapunov theory and stochastic Bellman theory. In particular, we show that asymptotic and geometric stability in probability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that can clearly be seen to be the solution to the steady state form of the stochastic Bellman equation, and hence, guaranteeing both stochastic stability and optimality.

Automated summary

This article explores the use of Lyapunov functions in evaluating nonquadratic cost functionals for Ito-type nonlinear stochastic difference equations, showing that the cost functional can be evaluated in closed-form when related to an underlying Lyapunov function ensuring asymptotic stability in probability. By analyzing discrete-time linear and nonlinear stochastic dynamical systems, as well as developing a stochastic optimal control framework, the study establishes connections between stochastic Lyapunov theory and stochastic Bellman theory to guarantee both stability and optimality in the closed-loop nonlinear system.