Real-time surrogate-assisted preprocessing of streaming sensor data

A new dynamic programming based algorithm for real-time simultaneous segmentation, compression, and C2 -smoothing of potentially infinite data streams - studied in the context of streaming sensor data - is shown to be both effective and (energy) efficient. Its three key elements - the cubic splinelet based adaptive search space, the objective function (combining two measures of approximation errors), and the two-phase search space reduction technique (combining segment length based pruning and surrogate based pruning) - are presented in detail. The output quality is measured in terms of the signal approximation accuracy and the corresponding compression ratio. The numerical results show that the new algorithm outperforms both reference algorithms for all test streams, in some cases-significantly (up to 32% smaller approximation errors and up to 41% higher compression ratios). Due to the effective search space auto-adaptation and special pruning techniques, in typical cases, this is obtained with significantly lower computational cost. The proposed algorithm can be applied to various domains including online compression and smoothing of streaming data coming from IoT devices, sensor networks, and sensors located in autonomous vehicles (cars, drones) and robots. The possible application areas also include real-time IoT analytics and embedded time-series databases.

Automated summary

This study presents a new dynamic programming algorithm for real-time simultaneous segmentation, compression, and smoothing of data streams. The algorithm is proven to be effective and energy efficient in the context of streaming sensor data. The key elements of the algorithm, including the adaptive search space, objective function, and search space reduction technique, are discussed in detail. Numerical results demonstrate that the new algorithm outperforms reference algorithms in terms of approximation accuracy and compression ratio. The proposed algorithm can be applied to various domains, such as IoT devices, sensor networks, and autonomous vehicles.