Constrained Markov Decision Process Modeling for Optimal Sensing of Cardiac Events in Mobile Health

Rapid advances in the smartphone, wearable sensing, and wireless communication provide an unprecedented opportunity to develop mobile systems for smart health management. Mobile cardiac sensing collects health-related data from individuals and enables the extraction of information pertinent to cardiac conditions. However, wireless sensors in ambulatory care settings operate on batteries. All-time sensing and monitoring will result in fast depletion of the battery in the mobile system. There is an urgent need to develop optimal sensing schemes that will reduce energy consumption while satisfying the requirements in the detection of cardiac events. In this article, we develop a constrained Markov decision process (CMDP) framework to optimize mobile electrocardiography (ECG) sensing under the constraint of the energy budget. We first characterize the cardiac states from ECG signals using the heterogeneous recurrence analysis. Second, we model the stochastic dynamics in cardiac processes as a continuous-time Markov chain (CTMC). Third, we optimize the ECG sensing through a CMDP framework under the constraint of energy budget. Finally, we validate and evaluate the performance of our CMDP policy in both simulation and real-world case studies. Experimental results demonstrate that the proposed CMDP policy significantly outperforms the traditional uniform and mean-time-to-event (MTTE) policies. Specifically, the error of state estimation is reduced by 34.0% in the real-world case study for energy-constrained sensing of cardiac events.

Automated summary

The article introduces a constrained Markov decision process (CMDP) framework to optimize mobile electrocardiography (ECG) sensing under the constraint of the energy budget, and evaluates its performance in energy-constrained sensing of cardiac events, showing superior results compared to traditional policies.