Journal
IFAC PAPERSONLINE
Volume 51, Issue 32, Pages 462-467Publisher
ELSEVIER SCIENCE BV
DOI: 10.1016/j.ifacol.2018.11.428
Keywords
Markov chains; stochastic control; impulse control; graduate control; quasivariational inequality,numerical solution
Categories
Funding
- Russain Science Foundation [14-50-00150]
Ask authors/readers for more resources
This work considers the continuous and impulse control of the finite state Markov chains in continuous time. In general continuous control governs the transition rates between the states of Markov chain (MC), so the instants and directions of the state changes are random. Meanwhile sometimes there is an urgent necessity to realise the transition which leads to immediate change of the state. Since such transitions need different efforts and produce different effects on the function of the MC itself, one can consider this situation as an impulse control, and if at the same time there is a possibility of graduate control we are coming to the problem of joint continuous and impulse control. In the article we develop the martingale representation of the MC governed by joint continuous and impulse control and give an approach to the solution of the optimal control problem on the basis of the dynamic programming equation. This equation has a form of quasivariational inequality which in the case of finite state MC can be reduced to the solution of the system of ordinary differential equation with one switching line. We give the proof of the verification theorem and find the numerical solution for problems with deterministic and stochastic impulse controls. (C) 2018, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available