Journal
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY
Volume 68, Issue 10, Pages 10018-10029Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TVT.2019.2937842
Keywords
Cloud computing; Resource management; Adaptation models; Adaptive systems; Quality of service; Quality of experience; Computational modeling; Semi-Markov Decision Process (SMDP); Reinforcement Learning (RL) Algorithm; Vehicular Cloud System; Neural-Network; Quality of Experience (QoE); Quality of Service (QoS)
Categories
Funding
- National Natural Science Foundation of China [61731017, 61571375, 61728108]
- Sichuan Science and Technology Program [2019YFG0088, 2017HH0083]
Ask authors/readers for more resources
In this paper, we propose a novel adaptive cloud resource allocation model based on Semi-Markov Decision Process (SMDP) and Reinforcement Learning (RL) algorithm in vehicular cloud system. The issue of adaptive resource allocation for vehicular request is formed as an SMDP in order to gain the dynamics of vehicular requests arrival and departure. An optimized decision is made to guarantee the Quality of Service (QoS) of the vehicular cloud system and the Quality of Experience (QoE) of the vehicular users as well as to maximize the total system reward of the vehicular cloud system in consideration of the balance between the vehicular cloud resource expense and the system income. Furthermore, to capture the mobility feature of the vehicular cloud system, we also apply a neural-network-based RL algorithm to resolve our proposed SMDP-based adaptive cloud resource allocation model. Firstly, we use a Planning algorithm to get the action values under certain state-action pairs, which are the initial samples to train the neural network. Then the RL is used to update the parameters of the neural network, train the neural network and adaptively improve the decision strategy. Subsequently, an adaptive vehicular cloud resource allocation scheme which can approach the optimal strategy is obtained without the knowledge of the distribution function of vehicular requests arrival and departure during the RL process. The simulation results show that our proposed adaptive cloud resource allocation model for vehicular cloud system can reduce the probability of delay in processing requests and achieve high system rewards in comparison with the regularly used greedy resource allocation method. The performance of the RL solution approaches that of traditional value iteration solution for our proposed adaptive cloud resource allocation model.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available