Optimal replacement of a system according to a semi-Markov decision process in a semi-Markov environment

This paper investigates an optimal replacement problem of a system in a semi-Markov environment. The system itself deteriorates according to a semi-Markov process, and is further influenced by its environment, which changes according to a semi-Markov process. Each change of the environment's state will change the parameters modelling the system and also cause damage on the system. For minimizing the discounted total costs with finite and infinite horizons, we show the existence of optimal control limit policies. A special case of Markov environment is discussed, and the state space can be simplified equivalently to be finite, so the real computation of the problem is feasible. Finally, a numerical example is given to prove the correctness and validity of the analysis.