期刊
IEEE TRANSACTIONS ON COMMUNICATIONS
卷 69, 期 12, 页码 8335-8351出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TCOMM.2021.3114681
关键词
Sensors; Batteries; Wireless sensor networks; Sensor systems; Monitoring; Minimization; Energy harvesting; Internet of Things (IoT); age of information (AoI); energy harvesting; reinforcement learning (RL); value iteration algorithm (VIA); dynamic programming; Q-learning
资金
- Infotech Oulu
- Academy of Finland [323698, 319485]
- Academy of Finland 6Genesis Flagship [318927]
- European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie Grant [793402]
- HPY Research Foundation
- Riitta ja Jorma J. Takanen Foundation
This research focuses on the freshness of information in IoT systems, optimizing the actions of edge nodes through RL algorithms to significantly reduce the average Age of Information (AoI) for users receiving measurements.
Information freshness is crucial for time-critical IoT applications, e.g., monitoring and control. We consider an IoT status update system with users, energy harvesting sensors, and a cache-enabled edge node. The users receive time-sensitive information about physical quantities, each measured by a sensor. Users demand for the information from the edge node whose cache stores the most recently received measurements from each sensor. To serve a request, the edge node either commands the sensor to send an update or retrieves the aged measurement from the cache. We aim at finding the best actions of the edge node to minimize the average AoI of the served measurements at the users, termed on-demand AoI. We model this problem as a Markov decision process and develop reinforcement learning (RL) algorithms: model-based value iteration and model-free Q-learning. We also propose a Q-learning method for the realistic case where the edge node is informed about the sensors' battery levels only via the status updates. The case under transmission limitations is also addressed. Furthermore, properties of an optimal policy are characterized. Simulation results show that an optimal policy is a threshold-based policy and that the proposed RL methods significantly reduce the average cost compared to several baselines.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据