Journal
INFORMATION SYSTEMS
Volume 105, Issue -, Pages -Publisher
PERGAMON-ELSEVIER SCIENCE LTD
DOI: 10.1016/j.is.2021.101953
Keywords
Fog computing; Optimization; Sensors; Data quality
Categories
Funding
- Hellenic Foundation for Research and Innovation [1052]
Ask authors/readers for more resources
Our work aims to address the increasing demand for performing complex analytics jobs close to edge devices, while considering data quality as an optimization objective. We propose a hybrid solution that improves global task allocation by trading off latency and extent of quality checks, showing significant improvement over existing solutions in experimental evaluations.
Our work is motivated by the fact that there is an increasing need to perform complex analytics jobs over streaming data as close to the edge devices as possible and, in parallel, it is important that data quality is considered as an optimization objective along with performance metrics. In this work, we develop a solution that trades latency for an increased fraction of incoming data, for which data quality-related measurements and operations are performed, in jobs running over geo-distributed heterogeneous and constrained resources. Our solution is hybrid: on the one hand, we perform search heuristics over locally optimal partial solutions to yield an enhanced global solution regarding task allocations; on the other hand, we employ a spring relaxation algorithm to avoid unnecessarily increased degree of partitioned parallelism. Through thorough experiments, we show that we can improve upon state-of-the-art solutions in terms of our objective function that combines latency and extent of quality checks by up to 2.56X. Moreover, we implement our solution within Apache Storm, and we perform experiments in an emulated setting. The results show that we can reduce the latency in 86.9% of the cases examined, while latency is up to 8 times lower compared to the built-in Storm scheduler, with the average latency reduction being 52.5%. (C) 2021 Published by Elsevier Ltd.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available