Journal
SIAM JOURNAL ON CONTROL AND OPTIMIZATION
Volume 53, Issue 3, Pages 1569-1598Publisher
SIAM PUBLICATIONS
DOI: 10.1137/140969221
Keywords
Markov decision processes; stochastic optimization; risk measures; conditional value-at-risk; stochastic dominance constraints; convex analytic approach
Categories
Funding
- Office of Naval Research Young Investigator Award [N000141210766]
- National Science Foundation CAREER Award [0954116]
- Direct For Computer & Info Scie & Enginr
- Division Of Computer and Network Systems [0954116] Funding Source: National Science Foundation
Ask authors/readers for more resources
In classical Markov decision process (MDP) theory, we search for a policy that, say, minimizes the expected infinite horizon discounted cost. Expectation is, of course, a risk neutral measure, which does not suffice in many applications, particularly in finance. We replace the expectation with a general risk functional, and call such models risk-aware MDP models. We consider minimization of such risk functionals in two cases, the expected utility framework, and conditional value-at-risk, a popular coherent risk measure. Later, we consider risk-aware MDPs wherein the risk is expressed in the constraints. This includes stochastic dominance constraints, and the classical chance-constrained optimization problems. In each case, we develop a convex analytic approach to solve such risk-aware MDPs. In most cases, we show that the problem can be formulated as an infinite-dimensional linear program (LP) in occupation measures when we augment the state space. We provide a discretization method and finite approximations for solving the resulting LPs. A striking result is that the chance-constrained MDP problem can be posed as an LP via the convex analytic method.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available