Optimal Selective Transmission Policy for Energy-Harvesting Wireless Sensors via Monotone Neural Networks

We investigate the optimal transmission policy for an energy-harvesting wireless sensor node. The node must decide whether an arrived packet should be transmitted or dropped, based on the packet's priority, wireless channel gain, and the energy status of the node. The problem is formulated under the Markov decision process (MDP) framework. For such a problem, the conventional method to get the optimal policy is by using a state value function, which is three-dimensional in the considered problem, leading to high complexity. Fortunately, to reduce complexity, we derive an equivalent solution for the optimal policy via a one-dimensional after-state value function. We show that the after-state value function is differentiable and nondecreasing. We also discover a threshold structure of the optimal policy that is derived by the after-state value function. Furthermore, to approximate the after-state value function, we propose a learning algorithm to train a three-layer monotone neural network. The trained network thus finds a near-optimal selective transmission policy of the node. Finally, through simulation, we demonstrate the learning efficiency of the algorithm and the performance of the learned policy.